At Storm ID, we have a list of Quality Assurance (QA) tickets that are included in the delivery phase of every project. The list may be lengthy, but everything in it is important for ensuring that we deliver the best possible site and that nothing is missed or forgotten.
These QA tickets include things like:
- 500 error page
- robots.txt file
- canonical links
- xml sitemap setup
- site not in debug mode
This is just a small sample of the QA tickets. While almost all of them are completed before the QA checklist is reviewed, it is still a time consuming and manual process to go through each ticket, check it, and mark it off as done.
We were looking for a way to automate our QA checking for Umbraco projects and then along came Umbraco 7.5 and, with it, Umbraco Health Checks.
What are Umbraco Health Checks?
Umbraco Health Check is a dashboard provided in the developer side of an Umbraco site build. This dashboard highlights 14 different checks which are spread over six categories:
- Data Integrity
- Live Environment
Developers use the Umbraco Health Check dashboard to ensure that your website is configured in alignment with best practices.
Running these Health Checks on your website will return a message detailing whether a problem was found in the code.
These built-in checks already cover some of our existing QA tickets such as Inline Macro Errors, Debug Compilation Mode, and Trace Mode.
You can access the full list of Health Checks within the official documentation.
However, we still have many of our own QA tickets that are not covered under Umbraco’s set of health checks. Fortunately, Umbraco allows you to add in new Health Checks within the Developer tab.
Storm QA Health Checks
Umbraco provides documentation to help you get started with writing your own Checks. There are two different methods for creating them: Configuration Checks and General Checks.
Each new Health Check is a class with a HealthCheck attribute attached. The attribute contains some basic information: a unique ID, a name, a short description, and a group name for the Health Check to sit in.
We started with a group called Storm QA and then began adding in a bunch of our own checks.
Configuration vhecks are nice and easy to implement and they allowed us to quickly target a number of our QA tickets. We inherit from AbstractConfigCheck and then the class uses XPath to check for the existence and value of a configuration setting. This can be within the Web.config (Web.Release.config, etc.), or the files within the Config folder (i.e. umbracoSettings.config).
Some of the checks that we have added include:
- Ensure that the error404 value in umbracoSettings.config is set (and it has not been left at the default value on 1)
- Check that the default Umbraco URL Tracker is disabled
- Values are set for Blob storage and image caching
- Google Maps API keys
Just as important as the above, we have further checks running on the web.release.config and web.debug.config files. These make sure that connection strings, API keys and general key values are removed before they are deployed to QA and production environments. We don’t want developer configuration to make its way to the QA and production environments.
Configuration checks also allow you to have a default value for settings, with an option for the administrator to “Fix” any issues. When using this feature, Umbraco will give the administrator the option to re-write the setting if it does not match what is expected. While this feature is good, we have decided not to use it. Our main reasoning for this decision was that it could have resulted in changes being made to settings for the production site that haven’t been tested locally by a developer, or checked into source control. The decision was taken to have the warning in place, and then let the developer fix the issue locally and test before deployment.
A General Check takes a little more setup, but is flexible in allowing the developer to create any text they choose. Once again, there is a great example within the Umbraco documentation, and this time you inherit from Umbraco.Web.HealthCheck.HealthCheck.
We have implemented some General Checks to ensure that canonical links are correct within the site. The check will look at the Home Page and the Landing Page (both with and without query string parameters) to see if the canonical links match the expected values.
We check for the existence of a 500.html error page (both the actual file in the folder and hitting the page with a web request).
We have other General Checks running and the list is being added to all the time.
We are still in the process of incorporating other QA tickets into Storm QA Health Checks. There are also other plans to see if we could start using it to check content within a site. Could we use it to check and report back on pages that don’t have OG or Twitter Card metadata defined?
As good as the Health Checks are, previous versions of Umbraco still relied on a user accessing the Developer Dashboard and clicking a button to run them. Umbraco 7.7, however, has the added benefit of Scheduled Health Checks. This will allow Umbraco to send out an email or push Health Check results to a Slack channel. While there is no admin interface to configure these scheduled checks, Umbraco has provided a method to configure this via a new HealthChecks.config file.
Configuring Health Checks to run after every deploymenet means we can get rid of the manual QA task to run the checks.
The future of developing websites in Umbraco, in any case, looks bright. We’re continually reviewing and revising our processes here at Storm ID to get the most from the software we’re using and Umbraco is no different. It will certainly be interesting to see the scope of Umbraco’s Health Check capabilities as we continue to further incorporate our own checks and tests going forward.