A (Fairly) Complete Guide to Performance Budgets [SmashingConf SF 2023]

Transcript

1. A (Fairly) Complete Guide
   to Performance Budgets
   @tameverts
   #SmashingConf
   

   View Slide

2. @tameverts
   webperf.social/@tammy
   

   View Slide

3. @tameverts
   

   View Slide

4. speedcurve.com/benchmarks
   

   View Slide

5. speedcurve.com/newsletter/
   

   View Slide

6. View Slide

7. Improved average load time from 6s à 1.2s
   7-12% increase in conversion rate + 25% increase in PVs
   Average load time degraded to 5s
   User feedback: “I will not come back to this site again.”
   Re-focused on performance
   0.4% increase in conversion rate
   2010
   2011
   2009
   @tameverts
   

   View Slide

8. 1. Constant feature development
   2. Badly implemented third-parties
   3. Waited too long to tackle problems
   4. Relied on performance sprints
   5. Stopped doing front-end performance measurement
   6. No way to track regressions
   @tameverts
   

   View Slide

9. View Slide

10. View Slide

11. timkadlec.com/2013/01/setting-a-performance-budget/
    

    View Slide

12. csswizardry.com/2020/01/performance-budgets-pragmatically/
    

    View Slide

13. speedcurve.com/customers/tag/performance-budgets/
    

    View Slide

14. “Fighting regressions took priority over
    optimizations. The reason we decided this was
    because in the past, when we had performance
    efforts, engineers who were working on
    optimizations couldn’t really see progress in our
    performance metrics, because there were so
    many regressions happening at the same time.”
    Michelle Vu, Pinterest
    perfnow.nl/2018#michelle
    

    View Slide

15. View Slide

16. What is a performance budget?
    Which metrics should I focus on?
    What should my budget thresholds be?
    How can I stay on top of my budgets?
    @tameverts
    

    View Slide

17. What is a performance budget?
    

    View Slide

18. Thresholds YOU create for metrics
    that are meaningful for YOUR site
    Time-based • Start Render, Largest Contentful Paint
    Quantity-based • Page size, image weight, Long Tasks
    Score-based • Cumulative Layout Shift, Lighthouse scores
    @tameverts
    

    View Slide

19. Monitoring tools
    Synthetic (lab)
    Mimics defined network & browser conditions
    No installation required
    Limited URLs
    Limited test locations
    Compare any sites
    Detailed analysis & visuals
    Real user monitoring (field)
    Real network & browser conditions
    Requires JavaScript installation
    Large sample size (up to 100%)
    Geographic spread
    Only measure your own site
    Correlation with other metrics (e.g., bounce rate)
    @tameverts
    

    View Slide

20. View Slide

21. A good performance budget
    shows you…
    q What your budget is
    q When you go out of bounds
    q How long you’re out of bounds
    q When you’re back within budget
    @tameverts
    

    View Slide

22. Budgets can be passive (e.g. charts)
    Get alerts so you can investigate
    Even break the build
    

    View Slide

23. budget
    exceeded fixed
    @tameverts
    oops
    

    View Slide

24. Which metrics
    should I focus on?
    

    View Slide

25. View Slide

26. TTFB DNS FCP FMP
    LCP CLS FID TBT
    TTI INP OMG WTF
    

    View Slide

27. View Slide

28. 50+ synthetic metrics
    30+ RUM metrics
    25 additional metrics for EACH third party
    ∞ custom metrics
    

    View Slide

29. MVB
    

    View Slide

30. MVB
    Minimum Viable Budget
    

    View Slide

31. Is the page loading?
    Can I use it?
    How does it feel?
    

    View Slide

32. ❑ Tracks the most important content
    ❑ Is accessible out of the box
    ❑ Is available in synthetic and RUM
    ❑ Can be correlated to UX and business metrics
    ❑ Has broad browser support
    The ideal UX metric…
    @tameverts
    

    View Slide

33. Is the page loading?
    

    View Slide

34. Backend time
    The time from the start of the initial navigation
    until the first byte is received by the browser
    (AKA Time to First Byte)
    Synthetic & RUM
    

    View Slide

35. View Slide

36. ❑ Tracks the most important content
    ❑ Is accessible out of the box
    ❑ Has broad browser support
    ❑ Is available in synthetic and RUM
    ❑ Can be correlated to UX and business metrics
    √
    √
    √
    Backend time
    √
    

    View Slide

37. Start Render
    The time from the start of navigation
    until the first non-white content is painted
    Synthetic & RUM
    

    View Slide

38. View Slide

39. As Start Render improves, so does bounce rate (typically)
    @tameverts
    

    View Slide

40. So does conversion rate (again, typically)
    @tameverts
    

    View Slide

41. ❑ Tracks the most important content
    ❑ Is accessible out of the box
    ❑ Has broad browser support
    ❑ Is available in synthetic and RUM
    ❑ Can be correlated to UX and business metrics
    √
    √
    √
    Start Render
    √
    √
    @tameverts
    

    View Slide

42. Can I use the page?
    

    View Slide

43. 43
    When do users start interacting with a page?
    @tameverts
    

    View Slide

44. Largest Contentful Paint (LCP)
    Amount of time it takes for the largest visual
    element (image or video) to render
    Synthetic & RUM
    (Chromium browsers only)
    

    View Slide

45. View Slide

46. Largest Contentful Paint (often) correlates to bounce rate
    @tameverts
    

    View Slide

47. ❑ Tracks the most important content
    ❑ Is accessible out of the box
    ❑ Has broad browser support
    ❑ Is available in synthetic and RUM
    ❑ Can be correlated to UX and business metrics
    √
    √
    Largest Contentful Paint
    √
    √
    @tameverts
    

    View Slide

48. Last Painted Hero (LPH)
    When the last piece of critical content
    (hero image, first H1) is painted in the browser.
    Synthetic
    

    View Slide

49. View Slide

50. ❑ Tracks the most important content
    ❑ Is accessible out of the box
    ❑ Has broad browser support
    ❑ Is available in synthetic and RUM
    ❑ Can be correlated to UX and business metrics
    √
    √
    Last Painted Hero
    √
    @tameverts
    

    View Slide

51. How does the page feel?
    

    View Slide

52. Cumulative Layout Shift (CLS)
    Aggregate score that reflects how much
    page elements shift during rendering
    Synthetic & RUM
    (Chromium browsers only)
    

    View Slide

53. View Slide

54. Bounce rate gets worse as CLS degrades
    Bounce rate improves as CLS degrades
    Bounce rate stays the same as CLS degrades
    speedcurve.com/blog/google-cumulative-layout-shift/
    

    View Slide

55. ❑ Tracks the most important content
    ❑ Is accessible out of the box
    ❑ Has broad browser support
    ❑ Is available in synthetic and RUM
    ❑ Can be correlated to UX and business metrics
    √
    √
    Cumulative Layout Shift
    @tameverts
    √
    

    View Slide

56. Long Tasks
    Any JavaScript function that takes >50ms to execute
    Long Tasks don’t always block page rendering,
    but they can cause the page to feel janky
    Synthetic & RUM
    

    View Slide

57. 57
    

    View Slide

58. speedcurve.com/blog/first-input-delay-google-core-web-vitals/
    Long Tasks Time often correlates to conversions
    @tameverts
    

    View Slide

59. 59
    Long Tasks Time can expose issues with third parties
    @tameverts
    

    View Slide

60. ❑ Tracks the most important content
    ❑ Is accessible out of the box
    ❑ Has broad browser support
    ❑ Is available in synthetic and RUM
    ❑ Can be correlated to UX and business metrics
    √
    Long Tasks
    @tameverts
    √
    √
    √
    

    View Slide

61. Interaction to Next Paint (INP)
    Measures a page's responsiveness
    to individual user interactions
    Observes latency and reports a single value
    that all (or nearly all) interactions are below
    RUM
    

    View Slide

62. speedcurve.com/blog/interaction-to-next-paint-core-web-vitals/
    

    View Slide

63. ❑ Tracks the most important content
    ❑ Is accessible out of the box
    ❑ Has broad browser support
    ❑ Is available in synthetic and RUM
    ❑ Can be correlated to UX and business metrics
    √
    Interaction to Next Paint
    @tameverts
    √
    √
    

    View Slide

64. Meaningful
    content
    Usable out of
    the box
    Broad browser
    support
    Synthetic RUM
    Correlates to
    business/UX
    Time to First Byte ★ ★ ★ ★ ☆
    Start Render ☆ ★ ★ ★ ★ ★
    Largest Contentful Paint ☆ ★ ★ ★ ★
    Last Painted Hero ☆ ★ ★ ★
    Cumulative Layout Shift ☆ ☆ ★ ★
    Long Tasks ★ ★ ★ ★ ☆
    Interaction to Next Paint ★ ★ ★ ★
    @tameverts
    

    View Slide

65. Page weight & complexity
    

    View Slide

66. View Slide

67. @tameverts
    

    View Slide

68. YMMV
    

    View Slide

69. What should
    my budget thresholds be?
    

    View Slide

70. performance budgets
    ≠
    performance goals
    

    View Slide

71. Goals are aspirational
    How fast do I want to be eventually?
    Budgets are pragmatic
    How can I keep my site from getting slower
    while I work toward my goals?
    @tameverts
    

    View Slide

72. View Slide

73. View Slide

74. X
    

    View Slide

75. demoralizing not actionable ignorable
    

    View Slide

76. As you optimize your pages
    (and hopefully get faster!)
    update your budgets
    

    View Slide

77. How do I manage
    my budgets?
    

    View Slide

78. Should I set up budgets
    in RUM, synthetic, or both?
    

    View Slide

79. Scenario 1
    Create budgets in RUM,
    and drill down via synthetic
    

    View Slide

80. @tameverts
    

    View Slide

81. Scenario 2
    Create budgets in synthetic,
    and integrate with CI/CD
    

    View Slide

82. @tameverts
    

    View Slide

83. Should I create
    performance budgets
    for every metric I track?
    

    View Slide

84. View Slide

85. View Slide

86. View Slide

87. Do I need performance budgets
    for all my pages?
    

    View Slide

88. 1. Product
    2. Category
    3. Home
    4. Cart
    5. Search
    @tameverts
    

    View Slide

89. Who should be responsible
    for managing budgets?
    

    View Slide

90. View Slide

91. Everyone* who touches a page
    should understand the performance
    impact of their choices
    *Yes, this includes marketing people
    

    View Slide

92. For example…
    If your marketing team is responsible for adding
    and maintaining third-party tags, they should:
    q Have a basic understanding of the metrics (such as Long Tasks Time)
    q Collaborate on setting the budget
    q Receive alerts when the budget is violated
    q Participate (or at least have visibility) in identifying and fixing the issue
    

    View Slide

93. View Slide

94. 1. Start small (even a single metric will do!)
    2. Visually validate your metrics (filmstrips, videos)
    3. Validate your metrics some more (UX, business)
    4. Get buy-in from different stakeholders
    5. Focus on the pages that matter most
    6. Revisit your budgets regularly (2-4 weeks)
    7. Remember that metrics are always evolving
    8. Never stop measuring
    

    View Slide

95. Thank you!
    @tameverts
    webperf.social/@tammy
    

    View Slide

96. A Complete Guide to Performance Budgets
    speedcurve.com/blog/performance-budgets/
    Setting a Performance Budget
    timkadlec.com/2013/01/setting-a-performance-budget/
    Performance Budgets, Pragmatically
    csswizardry.com/2020/01/performance-budgets-pragmatically/
    Web Vitals
    web.dev/vitals/
    Farewell FID… hello Interaction to Next Paint
    speedcurve.com/blog/interaction-to-next-paint-core-web-vitals/
    Cumulative Layout Shift: What it measures (and what it doesn’t)
    speedcurve.com/blog/google-cumulative-layout-shift/
    

    View Slide