The Linux Foundation is a non-profit consortium dedicated to fostering the growth of Linux.
|
|
Book/Distros
From The Linux Foundation
LSB Certifying for Linux Distributions
Introduction
Products or ranges of products that meet the LSB conformance requirements can apply for LSB certification. The certification process requires a demonstration of conformance to the LSB Certification Authority (a vendor neutral third party). The detailed requirements for conformance are identified in the LSB Product Standards and include passing sets of tests and completing a conformance statement. Only suppliers of certified products are permitted to use the LSB trademark in connection with their product. [1]
This chapter provides an overview of LSB Certification for a runtime environment and is based on the Guide to LSB Certification [2] produced by The Open Group. An overview of LSB Certification for application programs is provided in the next chapter. The first section of this chapter applies equally to both runtime environment and application certification
A number of documents contain the requirements of the LSB Certification program. The LSB Certification Policy governs the Free Standards Group LSB Certification Program. The use of the LSB trademark is governed by the LSB Trademark License Agreement. The LSB Certification Agreement covers the terms and conditions of the Certification Service. If you intend on submitting a product for certification then you must agree to the terms and conditions outlined in all three documents prior to certification.
Notes
|
[1] |
Portions of this chapter were derived with permission from LSB Certification, The Open Group. |
|
[2] |
Achieving LSB Certification
This section details the steps required to achieve certification, and provides information on how to perform each of these steps. The process involves:
- Understanding the certification program and process
- Testing the product informally
- Applying for certification
- Formal testing and submission of results
Understanding the Certification Program
To become familiar with the program, the following program documents should be read:
- LSB Certification Policy
- LSB Product Standards
LSB Certification Policy
It is best to start with the LSB Certification Policy, which is the foundation of the program. It provides information on the types of products that can be certified in the program, what it means to be certified, what is required to get a product certified and how to make sure that a product remains certified. The LSB Certification Policy defines the following types of products:
- LSB Application
- LSB Runtime Environment
- LSB Internationalized Runtime Environment
- LSB Build Environment
The definitions of these products are included in Section 1, Overview of the LSB Certification Policy.
|
| Note |
|---|---|
|
Registrations are currently being accepted for the first three of these products in this version of the program. |
The LSB Product Standards
The LSB Product Standards should then be read to understand the detailed conformance requirements against which a product can be certified. Product standards provide a mapping between certification, the LSB specifications, and the test suites needed to demonstrate conformance. There is an LSB product standard for each type of product that can be certified. The LSB Certification Web site should be consulted for the current set of product standards for which certifications are being processed.
In order to determine which product standards are suitable for a product to be certification against, applicants should review Section 3, Conformance of the LSB Certification Policy and the LSB Product Standards
Confidentiality
The applicant's company and product details are confidential between them and the LSB Certification Authority. This is effective once the applicant accepts the LSB Certification Agreement, which is a prerequisite to registering a product. The Certification Authority does not make any certification information available to any third party, including Free Standards Group employees, consultants, or members, without the written permission of the applicant.
Upon successful completion of the certification process, the certified product will be included in the publicly available Certification Register. However, to enable applicants to launch a certified product, they can request that certification remain confidential for up to 6 months from the date of written notification by the Certification Authority that a product has achieved certification. A request that the certification remain confidential can be made in the final step of the product registration (see the Section called The Final Step--Confirming Your Submission). See Section 11.3 of the LSB Certification Policy for further information.
Informal Testing
Normally, testing a product with the LSB certification test suite(s) is the first activity. Applicants should read the applicable product standard for the product that is being considered for certification, referring to the Indicators of Compliance section to identify the required certification test suites. These test suites are freely available from the Free Standards Group and should be used, along with any additional method deemed appropriate, to make sure that the product meets the applicable conformance requirements and is ready for entry into the certification program.
Authorized Test Suites
It should be noted that for formal testing, only currently authorized versions of the LSB certification test suites can be used. So, for informal testing it is vital that you use an authorized version. The Certification Authority maintains a list of the currently authorized certification test suites. Since the test suites come from multiple sources this online document includes information on how to obtain the test suites.
Following is information on how to find further information on the test suites and where to report any bugs.
Problems and Problem Reporting
If problems are encountered with running the certification test suites, first read the documentation provided with the test suite. If this does not answer the questions, please check the Frequently Asked Questions. If an answer still cannot be found, a report should be sent to the appropriate Test Suite Maintenance Authority as noted in the Frequently Asked Questions file.
If one of the authorized test suites is being used and failures occur but it is believed that the product conforms to the specifications, check the Problem Report information database. The online Problem Report database contains known information on existing bugs and interpretations of the specification. The issue may already be resolved by an LSB specification interpretation, or by an agreed Test Suite Deficiency. If it is not, please submit a new problem report.
Certification
Certification is a formal process. The supplier of a certified product, warrants and represents that the product meets all the conformance requirements applicable to the product standard against which it is certified, and implements all the features claimed in the Conformance Statement.
The obligations, terms and conditions of certification are fully set out in the following documents:
- LSB Certification Policy
- LSB Trademark License Agreement
- LSB Certification Agreement
These documents are available via the LSB Certification Web page. These documents should be read before the certification process is started in order to fully understand the policies and requirements.
The LSB Trademark License Agreement (TMLA)
The LSB Trademark License Agreement is between the Supplier and the Free Standards Group and requires signature. It is recommended that Suppliers commence the process to complete the Trademark License Agreement as soon as possible to save delays later in the certification process. A trademark license must be completed before a product can complete the certification process and be entered onto the Certification Register.
The Trademark License agreement only needs to be entered into once per organization. When it is in place multiple products can be certified.
The LSB Certification Agreement
You are required to agree to the LSB Certification Agreement for each product registration. The LSB Certification Agreement is between the Supplier and the Certification Authority. It defines the certification service and the legal commitment to the conditions of the service.
|
| Note |
|---|---|
|
The LSB Certification Agreement is a Web-based agreement. When Suppliers click the "I accept" button they indicate that they accept the terms and conditions. |
Product Registration
The First Step-- The Trademark License Agreement
Although not required to be completed before commencing with product registration, we recommend Suppliers to submit a completed LSB Trademark License Agreement to the Free Standards Group as early as possible in the process.
The Conformance Statement
The next step is to complete a Conformance Statement to describe your product and how it meets the conformance requirements. A Conformance Statement will be linked into the Certification Register entry for the product once it is certified. A template Conformance Statement Questionnaire is available for each Product Standard.
The LSB Certification Web site should be consulted for the list of the currently available Conformance Statement Questionnaires.
Suppliers must complete the relevant questionnaire to create a Conformance Statement for a product. See Sections 1.2, Section 2 and Section 3.3 of the LSB Certification Policy for further information on the purpose of the Conformance Statement.
Conformance Statement Questionnaires are provided as html templates. Suppliers should download the template and edit the html, retaining the existing layout. The completed Conformance Statement must then be submitted in html format to the Certification Authority using the Web certification system.
Conformance Statement Questionnaires have common frontmatter that ask for the following information about a company and the product being certified:
- Submitter Information
- Organization name and name of the author of the conformance statement.
- Product Information
- The name of the product and its unique version/release identifier.
- Operation Environment
- The LSB Processor Specific Architecture with which the conformance of the product has been demonstrated.
- Indicator(s) of Compliance
- The name and versions of the test suites used to demonstrate the conformance of the product.
- The Policy on Re-branding of This Product
- This question allows a Supplier to state its policy with respect to third parties re-branding and recertifying the product. If an open re-branding policy the Supplier can state here that it waives the requirement in the LSB Certification Policy that requires written permission to be given in this case. Otherwise, the Supplier might just want to indicate that the policy is as stated in the LSB Certification Policy.
The Conformance Statement Questionnaire also contains product standard specific questions that must be answered.
Submitting Initial Certification Information
The LSB Certification Authority provides a Web-based certification system through which products can be submitted for certification. Applying for certification requires the Supplier to submit some initial information via the Web. The initial submission involves completing a registration form (including company details and contact details), and accepting the terms and conditions of the LSB Certification program. This only has to be done for the initial product registration.
Consult the Certification Authority Web site for further information on the process of registering for use of the Certification Web system.
Registering Your Product within the Web Certification System
When a product is registered, a number of product details including the product name and version or release information for which conformance has been demonstrated have to be provided.
The Supplier must select the LSB Product Standard against which certification is being sought, can optionally enter a product related Web address and then must accept the LSB Certification Agreement in order to proceed with this product registration.
The product registration form is then displayed with additional fields for you to provide product standard specific information described in the following sections of this chapter.
LSB Runtime Specific Details
If a product is an LSB Runtime Environment the Supplier needs to select responses to the following questions about the product.
- Select the languages that the product supports.
- Select the region where the product is targeted.
- Select the hardware profile that the product requires (minimum memory, processor type, minimum disk space, minimum processor speed).
- Select the usage profile that best matches the product (desktop/server/cluster).
In several cases, multiple selections are allowed (for example, for the language and region). The responses to the previous list of questions will be made available in the Certification Register.
Uploading Your Conformance Statement
The certification system leads Suppliers through the steps required to register a product to be certified. The first document that must be uploaded to the certification system is the Conformance Statement for the product.
The next steps are to perform formal testing and then upload the test results as a sequence of files into the Web-based certification system.
Formal Testing for Certification
Formal testing is a self-test activity with results submitted to the Certification Authority for audit, via the certification system Web site. At this point in time you need to make sure you understand the required test campaign, that is, which test suites are required to be used for certification testing (see the following paragraph for details). You also need to make sure that you have obtained a currently authorized version of the certification test suite(s), since other versions will not be valid for formal testing.
Figure 1. LSB Test Campaign
When a test campaign requires warranty of the product working in a particular situation, or warranty that selected applications execute correctly on the product, then a Web form will prompt for confirmation during the final registration stage. For any problems encountered, it will allow for a problem report ID to be referenced.
You can obtain help and information as described in Section 6.2.3 of this chapter.
Test Journals
Test results submitted should always include the full journal output from an uninterrupted run of the applicable test suite. If this is not possible, a problem report must be raised stating the nature of the problem and you should wait for a response from the Certification Authority before proceeding further.
Within the certification Web system you are required to upload test journals for each applicable test suite. The system also allows you to obtain a summary report of the results status for a test journal. If you decide that this test journal is your formal certification test run for your product, you must then select the validation tool within the certification Web system. This will then automatically analyze the test results, and, if there are no failed test results or tests requiring manual resolution then the journal will be flagged as OK in the system. If there are failed test results detected by the validation tool, then a screen will be presented to you with a list of test cases that you will need to resolve. If there are tests requiring manual resolution a screen is also presented with a list of test cases you need to resolve.
The failed test results in the test journal that require resolution are those with a status of FAIL, UNINITIATED, UNRESOLVED, or UNREPORTED. The resolution process requires you to enter a valid reference to an approved problem report from the Problem Reporting database for each failed test case. An approved problem report is one that has resulted in an agreed Interpretation of the specification, a Test Suite Deficiency, or a Certification System Deficiency. These apply to specific releases of the specification, test suite, or certification system.
The test results in the journal that require manual resolution are denoted FIP (further information provided). The resolution process requires you to assert that the test case resolves to a PASS result for your system, and optionally to provide brief commentary. No reference is required to an approved problem report for FIP results.
Test Campaign Definition
For an initial certification a standard set of test campaigns are defined in the product standards. These are summarized in the following list:
- LSB Runtime Environment
- :* Completion of the LSB Runtime Environment Conformance Statement.
- Test Journal output (TET journal format required) from an uninterrupted run of the LSB runtime environment test suite.
- Test Journal output (TET journal format required) from an uninterrupted run of LSB Library Checker tool (lsblibchk).
- For the test journal output, any FAIL, UNRESOLVED, UNREPORTED, or UNINITIATED results need to be resolved by reference to associated granted interpretations, test suite deficiencies or certification system deficiences from the problem report database.
- For the test journal output, any FIP results need to be manually resolved to a PASS result.
- A list of the binary applications from the LSB Application Battery that the applicant warrants works correctly (Note that this declaration is provided in the conformance statement).
- LSB Internationalized Runtime Environment
- :* The product must be already or concurrently registered as an LSB Runtime Environment.
- Completion of the LSB Internationalized Runtime Environment Conformance Statement.
- Test Journal output (TET journal format required) from an uninterrupted run of the LSB Internationalized Runtime Environment automated test suite.
- For the test journal output, any FAIL, UNRESOLVED, UNREPORTED, or UNINITIATED results need to be resolved by reference to associated granted interpretations, test suite deficiencies, or certification system deficiences from the problem report database.
- For the test journal output, any FIP results need to be manually resolved to a PASS result.
- For the LSB Internationalized Runtime Environment interactive validation procedures completion of a self-assessment questionnaire within the Web certification system. This requires you to manually assert for each set of test assertions within the interactive validation procedures that you pass the assertion or reference a granted problem report.
For initial certifications a complete run of the listed test tools is required for the product standards listed. For renewals and certification updates you should refer to the Certification Authority Web site for more information.
The Final Step--Confirming Your Submission
Once input of all the required information is complete, the Web-based certification system displays a form into which the final details are entered to submit a product for certification. This is done in two stages:
- This form allows the Supplier to check all the information relating to the company, the product, and the registration. It is divided into sections; each one needs confirmation that it is correct. The form also includes the option to indicate that the registration be kept confidential prior to product launch. If any of the information needs correcting at this point, it can be updated.
- The second and final part is to affirm the commitment to the terms and conditions of the program as stated in the LSB Trademark License Agreement and the LSB Certification Agreement. Certification fees are due at this stage. Please note that if a payment method other than by credit card is chosen, the submission cannot be processed until the payment is received by the Certification Authority.
Completion of Certification
If the submission is complete and an LSB Trademark License Agreement is in place, the Supplier will be notified of successful certification within 15 calendar days. If for any reason the submission was not complete, the Supplier will be notified so that any corrections can be made and resubmitted. Such a resubmission needs to be done within 60 calendar days unless an extension has been granted by the Certification Authority. The initial fee covers one set of corrective actions.
The details of the certified product will then be put on the Certification Register, which is a public document, unless it has been requested that it remains confidential. To keep the certification confidential, this option must be selected in the Web certification system at the time the product is submitted.
When a product certification has been made public, the Supplier will be sent a certificate by electronic mail.
Certification is valid for a defined period as stated in Section 8.1, Duration of Certification of the LSB Certification Policy. At the end of that period, if the Supplier wishes the product to remain certified the certification of the product will need to be renewed. (See the section on Renewals and Certified Product Updates of this chapter.)
For product standards, such as the LSB Internationalized Runtime Environment, that are built upon other product standards the following rules apply. If registration is occurring at the same time as the other product standard then the duration of certification is the shorter of the two defined durations. Otherwise, if the registration references a pre-existing product standard certification for the product then the duration for the additional product standard is to the end-date associated with the existing certification.
Appeals Process
Suppliers may appeal decisions made by the Free Standards Group or the Certification Authority as stated in Section 10, Appeals Process of the LSB Certification Policy. Any request for appeal should be sent by electronic mail to lsb-cert@opengroup.org . Receipt of such a request will be acknowledged within 6 working days by the Certification Authority and the appeals process invoked.
Renewals and Certified Product Updates
The Renewals Process
A certified product has a defined period for its initial certification, after which it must be renewed or the product will no longer continue to be certified. The Certification Authority will notify the Supplier in about two months in advance by electronic mail when a renewal is due (it is important to keep contact information current). At any time, the Supplier may use the Web-based certification system to obtain the renewal dates for certified products.
Renewal implies that the product continues to conform and that the Supplier will continue to support the product for the duration of the renewal period. Consult the Certification Authority Web site for information on renewal policy and how renewals are undertaken.
Certified Product Updates
If any changes are made to a certified product, the Supplier may be required to retest or recertify as required by Section 6, Scope of Certification and Requirements for Product Retest and Recertification of the LSB Certification Policy. The Supplier should read that section thoroughly since it describes multiple scenarios related to product changes, some that require a certification related activity. In some cases a complete "Full certification" is required since the update is equivalent to a new product so far as certification is concerned.
Maintenance Releases
If the change to a product qualifies as a maintenance release (as defined in Section 6.1, Maintenance Releases of the LSB Certification Policy) the Supplier is not required to contact the Certification Authority.
Suppliers may request an update to a product's certification information to include the maintenance release. Depending on how the maintenance release is described, this may be a wise thing to do so as to avoid marketplace confusion regarding which versions of the products are certified. Updates to certification information of a product can be done by contacting the Certification Authority.
Renaming a Product
If a Supplier decides to rename a certified product (as defined in Section 6.3, Renamed Products of the LSB Certification Policy) they should contact the Certification Authority including the details of the required change and a written statement to the Certification Authority indicating that there have been no material changes to the certified product. The Certification Authority will change the product name on the Conformance Statement and in the Certification Register and issue a new certificate.
If a product is a submission in progress and has not been submitted for audit to the Certificaton Authority, then renaming the product can be done using the Web-based certification system.
Re-branding a Certified Product
If a Supplier wants to re-brand a product (as defined in Section 6.4, Re-branded products of the LSB Certification Policy), they should locate and read the Conformance Statement for that product to obtain information on the product suppliers policy with respect to re-branding. First, the Supplier should find out whether they have the permission required for re-branding. Then proceed into the Web-based certification system as if they were certifying a new product, except that when asked to input information about the product they should select that this is a "Re-branded" product.
The main difference between a "re-branded" product registration and a new product registration is that no test results are required to be submitted. Upon submission, the Certification Authority will verify that a Supplier has permission.
Currently supported systems
The LSB is one of the most successful ABI standards in the history of the Information Technology industry. Every major GNU/Linux distribution is LSB certified and LSB conformance testing has become a staple of quality assurance for everyone concerned. A current list of LSB certified distributions and applications can be found on the LSB Web site.