Live reporting
#
What you’ll learn
- What data are stored in which index
Canarytrace stores a lof of data to Elasticsearch indices. Elasticsearch index is in a simple analogous form something like a table in a SQL databases. For better work with data stores by Canarytrace runner it's necessary to setup Elasticsearch.
#
IndicesCanarytrace save continuously data to indices by type of data. Performance Audit, JS Heap, function test step report, all responses, performance entries etc.
c.report-*
contains test report data.c.performance-entries-*
contains data from performance entries API.c.request-log-*
contains all requests and responses called between browser and server.c.memory-*
contains continuously stores data about total and usage JS Heap.c.console-*
contains all messages from browser console.c.audit-*
contains data from Performance Audit (ligthhouse).c.coverage-audit-*
contains data from usage and unused resources loaded during parsing a web application.c.env-*
contains environment variables.c.smoke-title-*
contains URI with title content
c.report-*
#
Every action of Canarytrace runner fired event, which stored data about Canarytrace runner, tested app, and data from browser to Elasticsearch.
Search input with syntax for searching data in c.report-* index by uuid label
Time-range controll. Always use smaller range e.g. last 15min, or last 12 hour. The biggest range may affect the performance of the elasticsearch.
Selected index c.report-* and list of available labels in this index.
The bar graph with the occurence and amount of data received over time. If this graph is empty in your selected time range, probably Canarytrace runner is not running or your search input contains some search data, which aren't contains in this index. Next interesing information is count of records.
Table with received data from Canarytrace runner.
#
Eventsformated
c-report-*
index in Kibana, where's events name in event column
testStart
- Start Canarytrace runner, test case was successfully loaded, elasticsearch cluster is available, Canarytrace has enough resources to run.suiteStart
- This event corresponds with describe with name HomePage method and contains data available in this time. This event can be fired more times.testStepStart
- This event corresponds with it with name open method and contains data available in this time. This event can be fired more times.testStepEnd
- This event is fired when it is end and contains data with result of this test step. This event can be fired more times.testEnd
- This event is fired when is test case done.
event: testStart
uuid
is unique id of one running instance of Canarytrace runner. This id is the same for all events e.g. suiteStart, testStepStart etc. By uuid is possible search all events from one running test. The same uuid is stored in other Elasticsearch indices, so by this uuid is possible pairing data from c.report index and from c.performance-audit index.event
name of event which was fired.sequence
for every events, that is fired is assigned sequence, that is always incremented. It's useful for ordering stored events. E.g.testStart
has sequence 1suiteStart
has sequence 2testStepStart
has sequence 3labels
your optional labels, e.g.canary=smoke-pro
,pt.audit=desktopDense4G
etc.spec
path and name file with your test case, for Canarytrace Smoke Pro is spec always smoke.jstimestamp
is created in time, when Canarytrace fired event.
Labels seleniumCluster
, maxInstances
and goog:chromeOptions
are variables events and depends on your setup.
event: suiteStart
title
andfullTitle
are value from describe methods e.g.
Describe
and it
methods can tester change only for Canarytrace Professional, not in Canarytrace Smoke Pro
event: testStepStart
uuidAction
is unique id for test step / forit
method. Again, this is useful for searching in Kibana. The same uuidAction is stored in other Elasticsearch indices, so by this uuidAction is possible pairing data fromc.report
index and test step with name open and with e.g.c.performance-entries
index with record with the same uuidAction.It's useful for answer the question: Which performance.entries was loaded during open page?
title
is name of theit
methodfullTitle
is name of the parentdescribe
method
event: testStepEnd
testStepDuration
is time from firedtestStepStart
to firedtestStepend
in ms.errMessage
is description of error in case error or bug in test step. It can contains some critical error or value of assertation method e.g.expect(titleElm.waitForExist({timeoutMsg: "Element title not found. The page couldn't be loaded in time."})).to.be.true
errType
is type of error.passed
can contains value passed or falseattachments
in case error and if is optionATTACHMENTS=allow
is screenshot of web site stored on local storage and on AWS S3 storage. In this case this label contains path to screenshot.
event: testEnd
totalTestStepsDuration
is sum alltestStepDuration
in mstotalRunnerDuration
is total duration from Canarytrace runner inicializing to fired eventtestEnd
totalTestStepsDuration
andtotalRunnerDuration
can contains time with performance audit if you use this option.
c.performance-entries-*
#
Performance Entry is interface of Web APIs and provides access to performance-related information for the loaded/tested page. You can check how long does a file load e.g. your css, javascript, image, font or resources third-party e.g. Google Ads, Facebook pixel etc.
Canarytrace uses these API
- Performance Time API
- Navigation Timing API
- User Timing API
- Resource Timing API
and downloads performance entry type navigation
and resources
navigation
is always first init call from browser to infrastructure. For example, this method return how much time it takes to load or unload a document.resource
are further requests and their initiator is a parser. Is useful for analysis of detailed network timing data regarding the loading of an application's resources. An application can use the timing metrics to determine, for example, the length of time it takes to fetch a specific resource, such as anXMLHttpRequest
,<SVG>
, image, or script.
For example, open web site https://canarytrace.com/ and when is page loaded you can open DevTools > Network. You can se that navigation is canarytrace.com
and resource is some js
file.
First request is navigate type
Second request is resource type
Canarytrace downloads these collection after action open page, click or reload page in the monitor script. For one page can exists a lot of more performance entries in one collection. E.g. for home page of web site https://canarytrace.com/ contains performance entries collection 27 objects.
- Performance entries is automatically stores when is used PERFORMANCE_ENTRIES_INTERCEPT option and when is your Elasticsearch available. For more information go to CLI Options
Performance entries are intercepted by browser and Canarytrace downloads collection with performance entries when is fired event testStepEnd
. All performance entries intercepted between testStepStart
and testStepEnd
have the same uuidAction
such as testStepStart
and testStepEnd
. This is useful for searching what and how long it took during testStep open home page. E.g. what was downloaded to the browser and how long it took during opening home page tesla.com.
How to filter by entry type in Kibana
uuid : "d142f809-4909-47ac-8f30-eacb155ac38d"
is unique identificator for downloaded performance entries during test step.entryType: resource
is type of performance entry
entryType: navigation
Canarytrace added additional properties
sequence
is the order in the performance entries collection.labels
your optional labels, e.g.canary=smoke-pro
,pt.audit=desktopDense4G
etc.spec
path and name file with your test case, for Canarytrace Smoke Pro is spec always smoke.jstimestamp
is created in time, when Canarytrace fired eventtestStepEnd
fullTitle
is name of method describe + itresponseTime
is calculated fromresponseEnd
-requestStart
ttfb
is calculated fromresponseStart
-requestStart
entryType: resource
c.request-log-*
#
Requests and responses are collected from devTools.
Read more info about RequestLog service.
Request and response are paired by unique requestId
and sequence
is order of the request.
c.memory-*
#
Memory is collected from performance.memory
api.
- More info about Memory Intercept
jsHeapSizeLimit
The maximum size of the heap, in bytes, that is available to the context.totalJSHeapSize
The total allocated heap size, in bytes.usedJSHeapSize
The currently active segment of JS heap, in bytes.
c.console-*
#
Console events are collected directly from devTools.
- More info about Console Intercept
Type error of domain
xml
,javascript
,network
,storage
,appcache
,rendering
,security
,deprecation
,worker
,violation
,intervention
,recommendation
,other
c.audit-*
#
This index contains data from Lighthouse performance audit
c.coverage-audit-*
#
Collect coverage / used and unused of javascript code downloaded to browser.
- More info about Profiler.takePreciseCoverage
c.env-*
#
Collect all environment variables from POD before start Canarytrace.
danger
Print and store all environment variables including sensitive data if exist. You must use ENV_PRINT=allow
option for collect environment variable.
c.smoke-title-*
#
This index is created only via Canarytrace Smoke Pro and store reponse labels in check response phase from navigate request.
- Do you find mistake or have any questions? Please create issue, thanks 👍
- Have more questions? Contact us.