Article / Standard

TMH 18 data validation: from survey file to certificate

TMH 18 data validation explained: what version 4 of the standard defines, what TMH18 Validator accepts and checks, and what a certificate does and does not tell you.

Watch the overview

About 3 minutes. A narrated explainer; the same words are below.

Read the video text
  1. 0:00A road survey turns kilometres of road into rows of numbers.
  2. 0:05Before anyone can rely on those numbers, they have to arrive in a shared format, and they have to be checked.
  3. 0:12In South Africa, that shared format is TMH 18: Road Asset Data Electronic Exchange Formats.
  4. 0:22Version four, dated October 2014, lists thirteen file types, from the road network to visual condition, roughness and deflection.
  5. 0:34Each file is plain text. Every line is one record, and semicolons separate the fields.
  6. 0:41An empty field still keeps its place.
  7. 0:43But if a field is missing altogether, the standard says the file is rejected.
  8. 0:53Every reading is tied to a road, and to a kilometre position on that road.
  9. 0:58If a reading falls outside the road's kilometre range, the standard says it will be discarded.
  10. 1:05TMH 18 defines the format and a few integrity rules.
  11. 1:10It says nothing about certification, or about who signs a file off. That is the job TMH18 Validator does.
  12. 1:19You upload the file, and you choose a check template.
  13. 1:22The template decides which checks run. Nothing picks it for you, so choose the one that matches your survey.
  14. 1:30Every check ends in one of four results: passed, warning, error, or skipped.
  15. 1:36An error stops the file until it is fixed, or covered by a recorded exception.
  16. 1:42A warning asks you to look. And a skipped check did not run, so it is not a pass.
  17. 1:51If a flagged value is genuinely correct, someone with the right permission can record an exception, with a reason another engineer can check.
  18. 1:59If the data is wrong, fix the source and reupload. The checks run again, and earlier exceptions do not carry over.
  19. 2:08With no open errors, the file is sent for review and co-sign.
  20. 2:12Then a final certifier issues the certificate.
  21. 2:15That person must be someone other than the uploader, and other than the co-signer.
  22. 2:22The certificate stores a SHA-256 fingerprint of the exact file that was certified.
  23. 2:29Change a single character in that file, and the fingerprint no longer matches.
  24. 2:35A certificate records a review decision. It does not prove that the measurements are right.
  25. 2:40Anyone can look a certificate up on the public verifier, without signing in.
  26. 2:46So: prepare the file to the standard. Choose the right template. Resolve every error. And hand over the certificate with the exact file.
  27. 2:55The full guide is at TMH18 dot com, slash articles.

What does TMH 18 version 4 cover?

TMH 18, "Road Asset Data Electronic Exchange Formats", is a COTO standard published by SANRAL and dated October 2014. The copy this article uses is version 4, headed "Draft Version 4" on its inner title page and in every page footer.

It defines file formats, fields and a few integrity rules. It sets no condition thresholds and no survey method, and it says nothing about certification, sign-off or file hashes. This article follows what happens to a file after it is prepared.

Which files does TMH 18 list, and which does the upload accept?

Section 1.6 of version 4 says file extensions are case sensitive, and its Table 5 lists 13 of them. Between them they cover the road network, geometry, classification, lanes, traffic, visual condition, measured data and deflection.

The upload is a separate list, set by TMH18 Validator. It accepts .csv, the TMH 14 types .rsv and .rss, and 15 extensions that the product groups as TMH 18. Not all 15 come from version 4: .flx, .con, .blc and .vgi do not appear in its text. The geometry files .kml and .kmz are in version 4, but the upload does not accept them.

Do not read the two lists as one set. A file type that the upload accepts is not thereby a file type that the standard defines, and the reverse is also true.

How do TMH 18 data checks run, and what do the results mean?

You choose the Check template when you upload. TMH18 Validator does not detect it for you, and the checks that run are that template's. If the file extension differs from the template's, the checks still run and a warning is added. A .csv file never triggers this warning.

Nothing in the product links a check template to version 4 of the standard. A result therefore describes what that template tested. It does not describe the whole standard.

Results are counted as Errors, Warnings, Passed and Skipped. A skipped check did not run, so it is not a pass. The article on what errors, warnings and skipped checks mean explains how to read each one, and Check templates lists the templates you can choose from.

What happens after the checks?

Every signing step needs a completed check run. Review, co-sign and the final certificate follow, in that order. The guide from reviewed survey file to certificate gives the steps, and the article on when nobody can issue the final certificate explains who may certify.

What does a certificate record?

A certificate records a review decision and a SHA-256 fingerprint of the stored file. That fingerprint identifies the exact file that was certified, and changing the file produces a different one.

It does not prove that the measurements are right. Certification is a step in TMH18 Validator, not a requirement of the standard. The article on what a certificate tells the client who receives it covers how to read one.

  • Name the version of the standard you mean, and say "Draft Version 4" when it matters.
  • Treat the upload list and the standard's Table 5 as two separate lists.
  • Pick the Check template yourself, and know what it tests.
  • Account for every skipped check. It did not run.
  • Read a certificate as a decision and a fingerprint, not as proof of accuracy.

Put it into practice

Browse check templates

Sign-in and the relevant account permissions are required.