1000 JavaScript errors are as good as None

Transcript

1. 1000 JavaScript Errors are as
   good as None!
   Identifying the most important JavaScript Errors
   

   View Slide

2. Philip Tellis
   Principal RUM Distiller @ Akamai
   ● Analyses real user performance data from mPulse
   ● Author of the OpenSource boomerang RUM library
   twitter:@bluesmoon ⦿ github:@bluesmoon
   speakerdeck:@bluesmoon
   

   View Slide

3. Needles in a
   JavaScript
   Haystack(Trace)
   ★ Errors? What errors?
   ★ Too Much Information!
   ★ Ugh! Help me fix them 🙏
   

   View Slide

4. Hi all, Customer complains that we are
   blocking the “Fortsetzung ohne Akzeptanz”
   button on the website… they see this error
   

   View Slide

5. Developer:
   I can click on “Fortsetzung ohne
   Akzeptanz” and it works for me. I don’t
   see any error in the stack.
   

   View Slide

6. WFM!
   

   View Slide

7. User Machines are Strange Beasts!
   They are a mystery to us…
   

   View Slide

8. Boomerang!
   ● A JavaScript library to mainly collect performance
   data from the browser.
   ● It also collects information about JavaScript errors.
   ★ onerror global event handler
   ★ XMLHttpRequest responses that were not successful
   ★ Any calls to window.console.error
   ★ Runtime errors that happen during a callback for addEventListener,
   setTimeout and setInterval
   ★ And more!
   Note: Make sure cross-origin scripts have the crossorigin="anonymous" attribute and Access-Control-Allow-Origin header set.
   Code: https://github.com/akamai/boomerang Docs: https://techdocs.akamai.com/mpulse-boomerang/docs/errors
   

   View Slide

9. 25-120 million JavaScript
   errors every Day!
   

   View Slide

10. e=null; e.length
    Uncaught TypeError: Cannot read properties of null (reading 'length')
    TypeError: null is not an object (evaluating 'e.length')
    Unable to get value of the property 'length': object is null or undefined
    Uncaught TypeError: e is null
    

    View Slide

11. TypeError: A conexão de rede foi perdida
    TypeError: De netwerkverbinding is verbroken
    TypeError: Die Netzwerkverbindung wurde
    unterbrochen
    TypeError: La conexión de red se ha perdido
    TypeError: La conexión de red se perdió
    TypeError: La connexion réseau a été perdue
    TypeError: Mistet nettverksforbindelsen
    TypeError: Netværksforbindelsen gik tabt
    Grüß Gott Fehler, Salut erreur, नमस्ते गलती
    TypeError: Nätverksanslutningen förlorades
    TypeError: Połączenie sieciowe zostało
    utracone
    TypeError: Pripojenie k sieti bolo stratené
    TypeError: The network connection was lost
    TypeError: Сетевое соединение потеряно
    TypeError: ネットワーク接続が切れました。
    TypeError: 網路連線中斷。
    TypeError: 네트워크 연결이 유실되었습니다
    

    View Slide

12. TypeError: Abgebrochen
    TypeError: Kumottu
    TypeError: annullato
    TypeError: annulleret
    TypeError: annulé
    TypeError: anulat
    TypeError: avbruten
    TypeError: avbrutt
    TypeError: cancelado
    Kumottu, Abgebrochen, キャンセルしました
    TypeError: cancelled
    TypeError: geannuleerd
    TypeError: vazgeçildi
    TypeError: visszavonva
    TypeError: отменено
    TypeError: ถูกยกเลิก
    TypeError: キャンセルしました
    TypeError: 中止
    TypeError: 已取消
    

    View Slide

13. TypeError: A server with the specified hostname could not be found
    TypeError: Aucun serveur ayant le nom d’hôte précisé n’a pu être trouvé
    TypeError: Aucun serveur ayant le nom d’hôte précisé n’a été détecté
    TypeError: Es wurde kein Server mit dem angegebenen Hostnamen gefunden
    TypeError: The Internet connection appears to be offline
    TypeError: The internet connection appears to be offline
    Even different dialects
    

    View Slide

14. e=null; e…
    Cannot read properties of undefined (reading 'appendChild')
    Cannot read properties of undefined (reading 'childNodes')
    Cannot read properties of null (reading 'classList')
    Cannot read properties of null (reading 'innerHTML')
    Cannot read properties of null (reading '__vue__')
    Cannot read properties of null (reading 'add')
    Cannot read properties of null (reading 'addEventListener')
    

    View Slide

15. Script Error
    [object Object]
    zoid destroyed all components
    Error: Syntax error
    ReferenceError: 'google' is undefined
    window is not an object
    (empty string)
    Plugin errors
    🙀
    

    View Slide

16. DS 101: clean the data
    ● Different browsers reports the same error differently
    ● Errors are reported as text messages rather than numeric codes
    ● A certain browser also localizes messages based on browser
    locale.
    ● For any given issue, there will be many errors reported – one for
    each side-effect of that error.
    ● And then some errors are just useless.
    

    View Slide

17. Start in the Browser
    ● Browsers are consistent within the same page view
    ● We can pre-aggregate messages into groups before sending them
    to the server.
    Down to 10-30 million
    errors a Day 🎉
    

    View Slide

18. Step 2: Deal with the L10N
    ● Localized strings are tackled with a large mapping table.
    ● Normalizes all strings to their en-US equivalent.
    ● In some cases there are immutable parts, like variable identifiers
    that must remain unchanged, but change position.
    Down to 5-15 million
    errors a Day
    

    View Slide

19. Step 3: Common Error Names
    ● List of 300 regular expressions to munge errors down to a
    common format.
    ● Some of the regexes are chained.
    Down to 2-8 million
    errors a Day
    

    View Slide

20. Step 4: Uniqueness
    ● We can now look at unique errors for analysis, and what we see is
    surprising…
    Between 300-1700
    unique errors a Day 🎉
    

    View Slide

21. Copyright Randall Munroe https://xkcd.com/2343/
    

    View Slide

22. Clustering
    ● We have several parameters to cluster on.
    ● Filename & line number are useful.
    ● In most cases, the error text is the best indicator of its cluster.
    Down to 20-100 groups
    

    View Slide

23. Methods of Clustering
    ● Density-based spatial clustering of applications with noise
    (DBSCAN)
    ● K-Nearest-Neighbors (KNN)
    ● K-Means
    ● K-Medoids
    You’ll need to figure out a distance function for the dataset
    Martin Ester, Hans-Peter Kriegel, Jiirg Sander, Xiaowei Xu
    A density based approach for discovering Clusters in Large Spatial Databases with Noise
    

    View Slide

24. Error: Expected ')'
    Error: Expected ':'
    Error: Expected ';'
    Error: Expected '}'
    Error: Expected ']'
    Error: Expected '('
    Error: Expected identifier
    Error: Expected hexadecimal digit
    Error: Expected identifier, string or number
    Example Clusters
    Cannot read properties of null (reading '1')
    Cannot read properties of null (reading '0')
    Cannot read properties of null (reading 'add')
    Cannot read properties of null (reading 'blur')
    Cannot read properties of null (reading '7dvn')
    Cannot read properties of null (reading 'width')
    Cannot read properties of null (reading 'click')
    Cannot read properties of null (reading 'style')
    Cannot read properties of null (reading 'value')
    Cannot read properties of null (reading 'remove')
    Cannot read properties of null (reading '__vue__')
    Cannot read properties of null (reading 'getItem')
    Cannot read properties of null (reading 'setItem')
    Cannot read properties of null (reading 'nodeType')
    Cannot read properties of null (reading 'errorCode')
    Cannot read properties of null (reading 'startsWith')
    Cannot read properties of null (reading 'postMessage')
    Cannot read properties of null (reading 'removeChild')
    

    View Slide

25. Error Impact
    Identifying which errors have an impact
    

    View Slide

26. Which one do I fix first?
    ● Let’s look at errors that impact the user journey.
    ● Focus on errors that affect a user’s ability to convert.
    ● Can we tell if certain errors frustrate the user?
    ● What’s the performance impact of errors?
    

    View Slide

27. ↻
    Reload Rate
    ● Do users hit the reload button when this error
    occurs?
    ● Use the type property from the
    PerformanceNavigationTiming object.
    ● Make sure sample size is large enough,
    i.e. 100/500 reloads is more reliable than 3/6
    reloads even though it’s 20% v/s 50%
    

    View Slide

28. 😠
    Rage Click Rate
    ● Does the existence of a particular
    error correspond to a high number of
    Rage Clicks on the page with the
    error?
    

    View Slide

29. Rage Clicks occur when users rapid-fire click (or
    tap) on your site or app.
    Rage clicking is the digital equivalent of
    cursing to release frustration.
    https://www.fullstory.com/resources/guide-to-understanding-frustrating-user-experiences-online/
    https://www.psychologytoday.com/us/blog/hide-and-seek/201205/hell-yes-the-7-best-reasons-swearing
    Rage Clicks
    

    View Slide

30. 😠
    Rage Click Rate
    ● Does the existence of a particular
    error correspond to a high number of
    Rage Clicks on the page with the
    error?
    ● Points to user frustration
    

    View Slide

31. Truncated User Journey
    ● Some errors cause a user session to terminate immediately.
    ● Others seem to allow the user to proceed.
    ● It’s also instructive to split this report by converted and/or “might
    convert” status.
    

    View Slide

32. Errors affecting Performance
    ● Some errors make page load time worse.
    ● In this case inability to access localStorage probably
    requires extra network calls.
    Access denied for ‘localStorage’
    

    View Slide

33. Errors affecting Performance
    ● Some errors make page load time better.
    ● In some cases, it causes page load to fail resulting in partial or
    no-content.
    There was an error setting cookie .* Please check domain and path
    

    View Slide

34. If an error blocks content and makes
    the page faster, without reloads or
    truncated sessions, then is that
    content really needed?
    

    View Slide

35. Debugging Aids
    Information to identify root causes
    

    View Slide

36. How do I fix them?
    ● When did it start happening?
    related to a new product release, browser release, etc.
    ● Where does it happen?
    Browser, OS, specific page, Geo region, etc.
    ● What are the conditions when it happens?
    Specific scripts/css loaded/not loaded, load time, http/h2, etc.
    ● What was the user journey like when it happens?
    Previous page, SPA lifetime, Session Length
    

    View Slide

37. Errors over Time
    ● Look at total error count as well as average
    errors per page for a given error.
    ● Check if new spikes or fixes correlate with a new
    product release or a new browser release (we
    generally annotate our charts with product
    releases).
    ● For existing errors, look at trends in the average
    error count.
    

    View Slide

38. Where does the error happen?
    OS Browser Device
    The operation is insecure
    Country Page
    

    View Slide

39. Conditions corresponding to Errors
    Some errors only show up when particular
    JavaScript files fail to load… for example, due to an
    Ad-Blocker or high packet loss
    

    View Slide

40. Some errors show up when a particular JavaScript
    file loads up more than once!
    

    View Slide

41. I had the same error, it seems to have been caused by loading
    the PayPal script more than once. I fixed it by loading the script
    just once in the head, then loading the two buttons where they
    should be.
    zoid destroyed all components
    “
    RebeccaRey on github issues for paypal
    

    View Slide

42. Some errors show up when the page is proxied
    through Google Translate
    

    View Slide

43. CSRF token expired.
    Some errors often show up when particular
    resources were served from cache.
    

    View Slide

44. Some errors show up when the client device
    is in low power mode (battery < 10%).
    

    View Slide

45. Effects of User Journey
    ● Data corruption in cookies or localStorage might
    only show up after a user browses through a few
    pages.
    ● Memory leaks in Single Page Apps may only
    manifest as problems after several navigations.
    ● Cross-domain cookie issues could show up if the
    user had to pass through a different subdomain
    through their journey.
    🍪
    🥠
    

    View Slide

46. Dealing with Script Error
    

    View Slide

47. What’s the Problem?
    Script error 51,169,268
    [container] is not a valid DOM Element 2,845,511
    [object Object] 2,553,637
    ReferenceError: 'xxxxxx' is undefined 1,952,751
    Error: s:0 rs:4 1,445,856
    Attempting to change access mechanism for an unconfigurable property 1,083,304
    TypeError: [code] is not a function 809,376
    TypeError: undefined is not an object 789,812
    81%
    No information in Script Error
    

    View Slide

48. Why does this happen?
    ● Cross-origin script files that may be credentialed.
    ● The browser doesn’t know if error information may be sensitive or
    not, so it hides it by default.
    ● For example, an error that looks like this in the web dev console…
    

    View Slide

49. Looks like this in an onerror handler…
    ● No real error message, no filename, line number, stack, nothing.
    ● You only get a timestamp!
    

    View Slide

50. Two options to fix it
    1. If you have control over the page & script domain, do the
    following:
    a. Set the crossorigin="anonymous" attribute on the element.<br/>This tells the browser not to send cookies, and that this request is<br/>non-credentialed.<br/>b. Set the Access-Control-Allow-Origin HTTP response header on the<br/>script to either * or the domain of your host page.<br/>

    View Slide

51. Two options to fix it
    2. If you only have control over the page, do the following:
    ○ Wrap all your calls to 3rd party libraries in a try/catch block:
    try {
    libraryCall();
    }
    catch(e) {
    // e contains detailed information about the error
    }
    

    View Slide

52. ★ Collect user errors
    ★ Normalize & De-duplicate
    ★ Cluster similar errors
    ★ Measure User Impact
    ★ Handle Script Error
    Summary
    

    View Slide

53. Thank you!
    

    View Slide

54. ● boomerang error plugin: Source on github • Docs at Akamai
    ● null has no properties – MDN docs
    ● zoid destroyed all components – github issue
    ● A density based approach for discovering Clusters in Large Spatial Databases with
    Noise – Ester M, Kriegel H-P, Sander J, Xu X
    ● PerformanceNavigationTiming.type – MDN docs
    ● Rage Clicking – Fullstory
    ● Web Performance with Android's Battery Saver Mode – Goel U, Ludin S, Steiner M
    ● Side effects of boomerang’s JS Error tracking – Nic Jansma
    References
    

    View Slide