Conformance and compliance testing

Conformance testing is an element of conformity assessment, and also known as compliance testing, or type testing - is testing or other activities that determine whether a process, product, or service complies with the requirements of a specification, technical standard, contract, or regulation.

Web standards

QUIC: A UDP-Based Multiplexed and Secure Transport


HTML (HyperText Markup Language)






ECMAScript Language Specification - test262

RFC 2616: Hypertext Transfer Protocol – HTTP/1.1


RFC 7540: Hypertext Transfer Protocol Version 2 (HTTP/2)

There are two test suites: http2-test and h2spec.

RFC 4791: Calendaring Extensions to WebDAV (CalDAV)

RFC 4918: HTTP Extensions for Web Distributed Authoring and Versioning (WebDAV)

RFC 1738: Uniform Resource Locators (URL)

Hypothesis + lark-grammars

RFC 2396: Uniform Resource Identifiers (URI): Generic Syntax

Hypothesis + lark-grammars

RFC 2397: The “data” URL scheme

Hypothesis + lark-grammars

RFC 5545: Internet Calendaring and Scheduling Core Object Specification (iCalendar)

Hypothesis + lark-grammars

Robots Exclusion Protocol

A file robots.txt used by websites primarily to manage crawler traffic to site. For a long time format has differences in descriptions provided by Google and Yandex in their documentations for webmasters. For example Yandex describes an option “Crawl-delay” but there is no such option in documentation provided by Google. In 2019 Google had an attempt to formalize the Robots Exclusion Protocol Specification and they published a draft, that was expired in Jan 2, 2020. However proposed specification has a grammar that can be used for testing parsers and producers on robots.txt files. For such purposes once can use Hypothesis and lark-grammars or robotstxt-spec-test.

Structured Query Language (SQL)







Windows Protocol TestSuites

RDP (Remote Desktop Protocol)

Windows Protocol TestSuites



A Stateless Transport Tunneling Protocol for Network Virtualization (STT)


RFC 4960: Stream Control Transmission Protocol


Signaling System 7 (SS7)

  • RFC 4666: Signaling System 7 (SS7) Message Transfer Part 3 (MTP3) - User Adaptation Layer (M3UA)

  • RFC 3331: Signaling System 7 (SS7) Message Transfer Part 2 (MTP2) - User Adaptation Layer

  • RFC 3686: Signalling Connection Control Part User Adaptation Layer (SUA)

  • RFC 4165: Signaling System 7 (SS7) Message Transfer Part 2 (MTP2) - User Peer-to-Peer Adaptation Layer (M2PA)

  • m2pa-testtool

  • sua-testtool

  • m2ua-testtool

  • m3ua-testtool

RFC 3261: SIP: Session Initiation Protocol

TTCN-3 test suite |

RFC 1035: Domain Names - Implementation And Specification

DNS Compliance Tests


TCP (Transmission Control Protocol)


packetdrill testcases


SMTP smtpscript


Eclipse IoT-Testware

Data Formats

IEEE 754

There is a test suite for IEEE 754R compliance published by IBM. It contains test vectors arranged in files. Each file represents the test vectors for a model described in the document. Documentation, IBM Fpgen Floating-Point Test Generator.

There is a FPSpy, a tool that provide mechanisms for analyzing floating point arithmetic as it executes, making it possible to find and track problematic operations in unmodified x64 application binaries on Linux, including those using thread- and process-level parallelism. FPSpy can observe application behavior without any cooperation from the application or developer, and can potentially be deployed as part of a job launch process. Tool published as a part of research described in “Spying on the Floating Point Behavior of Existing, Unmodified Scientific Applications”. fpspy

Few C programs to test the floating-point arithmetic of your machine: Floating-Point Arithmetic Test Programs

Zoned Device ATA Command Set (ZAC)

RFC 822: Standard for the Format of ARPA Internet Text Messages

Hypothesis + lark-grammars

TAP (Test Anything Protocol)

TAP 13 TAP 14 Hypothesis + lark-grammars


YAML, Hypothesis + lark-grammars

RFC 8259: The JavaScript Object Notation (JSON) Data Interchange Format JSON Schema JSON-Schema-Test-Suite

TOML TOML toml-test


S3 compatibility tests


OPDS (Open Publication Distribution System) OPDS Test Catalog


Markdown is a simple widely-used text markup format. It was intended to be as easy-to-read and easy-to-write as is feasible. Initially it had a limited syntax, that was later was extended by additional syntax (like footnotes, math support etc). There are two testsuites that allows to test parsers and generators of texts with Markdown markup: mdtest and MarkdownTest.





There is a RISC-V compliance testsuite -

There is a MicroTesk. MicroTESK is a reconfigurable and extendable framework for generating test programs for functional verification of microprocessors. MicroTESK allows automatically constructing test program generators based on formal specifications of microprocessor architectures. MicroTESK supports a wide range of architectures including RISC, CISC, VLIW, and DSP. MicroTESK supports online test program generation.

USB (Universal Serial Bus)


Теги: softwaretestingfeeden