blob: 3b9190d47f4fc962847090b3a1b0c0a30e61ed52 [file] [log] [blame]
Andrew Geissler517393d2023-01-13 08:55:19 -06001.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
2
3Working With Licenses
4*********************
5
6As mentioned in the ":ref:`overview-manual/development-environment:licensing`"
7section in the Yocto Project Overview and Concepts Manual, open source
8projects are open to the public and they consequently have different
9licensing structures in place. This section describes the mechanism by
10which the :term:`OpenEmbedded Build System`
11tracks changes to
12licensing text and covers how to maintain open source license compliance
13during your project's lifecycle. The section also describes how to
14enable commercially licensed recipes, which by default are disabled.
15
16Tracking License Changes
17========================
18
19The license of an upstream project might change in the future. In order
20to prevent these changes going unnoticed, the
21:term:`LIC_FILES_CHKSUM`
22variable tracks changes to the license text. The checksums are validated
23at the end of the configure step, and if the checksums do not match, the
24build will fail.
25
26Specifying the ``LIC_FILES_CHKSUM`` Variable
27--------------------------------------------
28
29The :term:`LIC_FILES_CHKSUM` variable contains checksums of the license text
30in the source code for the recipe. Following is an example of how to
31specify :term:`LIC_FILES_CHKSUM`::
32
33 LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \
34 file://licfile1.txt;beginline=5;endline=29;md5=yyyy \
35 file://licfile2.txt;endline=50;md5=zzzz \
36 ..."
37
38.. note::
39
40 - When using "beginline" and "endline", realize that line numbering
41 begins with one and not zero. Also, the included lines are
42 inclusive (i.e. lines five through and including 29 in the
43 previous example for ``licfile1.txt``).
44
45 - When a license check fails, the selected license text is included
46 as part of the QA message. Using this output, you can determine
47 the exact start and finish for the needed license text.
48
49The build system uses the :term:`S`
50variable as the default directory when searching files listed in
51:term:`LIC_FILES_CHKSUM`. The previous example employs the default
52directory.
53
54Consider this next example::
55
56 LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\
57 md5=bb14ed3c4cda583abc85401304b5cd4e"
58 LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6"
59
60The first line locates a file in ``${S}/src/ls.c`` and isolates lines
61five through 16 as license text. The second line refers to a file in
62:term:`WORKDIR`.
63
64Note that :term:`LIC_FILES_CHKSUM` variable is mandatory for all recipes,
65unless the :term:`LICENSE` variable is set to "CLOSED".
66
67Explanation of Syntax
68---------------------
69
70As mentioned in the previous section, the :term:`LIC_FILES_CHKSUM` variable
71lists all the important files that contain the license text for the
72source code. It is possible to specify a checksum for an entire file, or
73a specific section of a file (specified by beginning and ending line
74numbers with the "beginline" and "endline" parameters, respectively).
75The latter is useful for source files with a license notice header,
76README documents, and so forth. If you do not use the "beginline"
77parameter, then it is assumed that the text begins on the first line of
78the file. Similarly, if you do not use the "endline" parameter, it is
79assumed that the license text ends with the last line of the file.
80
81The "md5" parameter stores the md5 checksum of the license text. If the
82license text changes in any way as compared to this parameter then a
83mismatch occurs. This mismatch triggers a build failure and notifies the
84developer. Notification allows the developer to review and address the
85license text changes. Also note that if a mismatch occurs during the
86build, the correct md5 checksum is placed in the build log and can be
87easily copied to the recipe.
88
89There is no limit to how many files you can specify using the
90:term:`LIC_FILES_CHKSUM` variable. Generally, however, every project
91requires a few specifications for license tracking. Many projects have a
92"COPYING" file that stores the license information for all the source
93code files. This practice allows you to just track the "COPYING" file as
94long as it is kept up to date.
95
96.. note::
97
98 - If you specify an empty or invalid "md5" parameter,
99 :term:`BitBake` returns an md5
100 mis-match error and displays the correct "md5" parameter value
101 during the build. The correct parameter is also captured in the
102 build log.
103
104 - If the whole file contains only license text, you do not need to
105 use the "beginline" and "endline" parameters.
106
107Enabling Commercially Licensed Recipes
108======================================
109
110By default, the OpenEmbedded build system disables components that have
111commercial or other special licensing requirements. Such requirements
112are defined on a recipe-by-recipe basis through the
113:term:`LICENSE_FLAGS` variable
114definition in the affected recipe. For instance, the
115``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` recipe
116contains the following statement::
117
118 LICENSE_FLAGS = "commercial"
119
120Here is a
121slightly more complicated example that contains both an explicit recipe
122name and version (after variable expansion)::
123
124 LICENSE_FLAGS = "license_${PN}_${PV}"
125
Andrew Geissler220dafd2023-10-04 10:18:08 -0500126It is possible to give more details about a specific license
127using flags on the :term:`LICENSE_FLAGS_DETAILS` variable::
128
129 LICENSE_FLAGS_DETAILS[my-eula-license] = "For further details, see https://example.com/eula."
130
131If set, this will be displayed to the user if the license hasn't been accepted.
132
Andrew Geissler517393d2023-01-13 08:55:19 -0600133In order for a component restricted by a
134:term:`LICENSE_FLAGS` definition to be enabled and included in an image, it
135needs to have a matching entry in the global
136:term:`LICENSE_FLAGS_ACCEPTED`
137variable, which is a variable typically defined in your ``local.conf``
138file. For example, to enable the
139``poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly`` package, you
140could add either the string "commercial_gst-plugins-ugly" or the more
141general string "commercial" to :term:`LICENSE_FLAGS_ACCEPTED`. See the
142":ref:`dev-manual/licenses:license flag matching`" section for a full
143explanation of how :term:`LICENSE_FLAGS` matching works. Here is the
144example::
145
146 LICENSE_FLAGS_ACCEPTED = "commercial_gst-plugins-ugly"
147
148Likewise, to additionally enable the package built from the recipe
149containing ``LICENSE_FLAGS = "license_${PN}_${PV}"``, and assuming that
150the actual recipe name was ``emgd_1.10.bb``, the following string would
151enable that package as well as the original ``gst-plugins-ugly``
152package::
153
154 LICENSE_FLAGS_ACCEPTED = "commercial_gst-plugins-ugly license_emgd_1.10"
155
156As a convenience, you do not need to specify the
157complete license string for every package. You can use
158an abbreviated form, which consists of just the first portion or
159portions of the license string before the initial underscore character
160or characters. A partial string will match any license that contains the
161given string as the first portion of its license. For example, the
162following value will also match both of the packages
163previously mentioned as well as any other packages that have licenses
164starting with "commercial" or "license"::
165
166 LICENSE_FLAGS_ACCEPTED = "commercial license"
167
168License Flag Matching
169---------------------
170
171License flag matching allows you to control what recipes the
172OpenEmbedded build system includes in the build. Fundamentally, the
173build system attempts to match :term:`LICENSE_FLAGS` strings found in
174recipes against strings found in :term:`LICENSE_FLAGS_ACCEPTED`.
175A match causes the build system to include a recipe in the
176build, while failure to find a match causes the build system to exclude
177a recipe.
178
179In general, license flag matching is simple. However, understanding some
180concepts will help you correctly and effectively use matching.
181
182Before a flag defined by a particular recipe is tested against the
183entries of :term:`LICENSE_FLAGS_ACCEPTED`, the expanded
184string ``_${PN}`` is appended to the flag. This expansion makes each
185:term:`LICENSE_FLAGS` value recipe-specific. After expansion, the
186string is then matched against the entries. Thus, specifying
187``LICENSE_FLAGS = "commercial"`` in recipe "foo", for example, results
188in the string ``"commercial_foo"``. And, to create a match, that string
189must appear among the entries of :term:`LICENSE_FLAGS_ACCEPTED`.
190
191Judicious use of the :term:`LICENSE_FLAGS` strings and the contents of the
192:term:`LICENSE_FLAGS_ACCEPTED` variable allows you a lot of flexibility for
193including or excluding recipes based on licensing. For example, you can
194broaden the matching capabilities by using license flags string subsets
195in :term:`LICENSE_FLAGS_ACCEPTED`.
196
197.. note::
198
199 When using a string subset, be sure to use the part of the expanded
200 string that precedes the appended underscore character (e.g.
201 ``usethispart_1.3``, ``usethispart_1.4``, and so forth).
202
203For example, simply specifying the string "commercial" in the
204:term:`LICENSE_FLAGS_ACCEPTED` variable matches any expanded
205:term:`LICENSE_FLAGS` definition that starts with the string
206"commercial" such as "commercial_foo" and "commercial_bar", which
207are the strings the build system automatically generates for
208hypothetical recipes named "foo" and "bar" assuming those recipes simply
209specify the following::
210
211 LICENSE_FLAGS = "commercial"
212
213Thus, you can choose to exhaustively enumerate each license flag in the
214list and allow only specific recipes into the image, or you can use a
215string subset that causes a broader range of matches to allow a range of
216recipes into the image.
217
218This scheme works even if the :term:`LICENSE_FLAGS` string already has
219``_${PN}`` appended. For example, the build system turns the license
220flag "commercial_1.2_foo" into "commercial_1.2_foo_foo" and would match
221both the general "commercial" and the specific "commercial_1.2_foo"
222strings found in the :term:`LICENSE_FLAGS_ACCEPTED` variable, as expected.
223
224Here are some other scenarios:
225
226- You can specify a versioned string in the recipe such as
227 "commercial_foo_1.2" in a "foo" recipe. The build system expands this
228 string to "commercial_foo_1.2_foo". Combine this license flag with a
229 :term:`LICENSE_FLAGS_ACCEPTED` variable that has the string
230 "commercial" and you match the flag along with any other flag that
231 starts with the string "commercial".
232
233- Under the same circumstances, you can add "commercial_foo" in the
234 :term:`LICENSE_FLAGS_ACCEPTED` variable and the build system not only
235 matches "commercial_foo_1.2" but also matches any license flag with
236 the string "commercial_foo", regardless of the version.
237
238- You can be very specific and use both the package and version parts
239 in the :term:`LICENSE_FLAGS_ACCEPTED` list (e.g.
240 "commercial_foo_1.2") to specifically match a versioned recipe.
241
242Other Variables Related to Commercial Licenses
243----------------------------------------------
244
245There are other helpful variables related to commercial license handling,
246defined in the
247``poky/meta/conf/distro/include/default-distrovars.inc`` file::
248
249 COMMERCIAL_AUDIO_PLUGINS ?= ""
250 COMMERCIAL_VIDEO_PLUGINS ?= ""
251
Andrew Geissler6aa7eec2023-03-03 12:41:14 -0600252If you want to enable these components, you can do so by making sure you have
253statements similar to the following in your ``local.conf`` configuration file::
Andrew Geissler517393d2023-01-13 08:55:19 -0600254
255 COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \
256 gst-plugins-ugly-mpegaudioparse"
257 COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \
258 gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse"
259 LICENSE_FLAGS_ACCEPTED = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp"
260
Andrew Geissler6aa7eec2023-03-03 12:41:14 -0600261Of course, you could also create a matching list for those components using the
262more general "commercial" string in the :term:`LICENSE_FLAGS_ACCEPTED` variable,
263but that would also enable all the other packages with :term:`LICENSE_FLAGS`
Andrew Geissler517393d2023-01-13 08:55:19 -0600264containing "commercial", which you may or may not want::
265
266 LICENSE_FLAGS_ACCEPTED = "commercial"
267
268Specifying audio and video plugins as part of the
Andrew Geissler6aa7eec2023-03-03 12:41:14 -0600269:term:`COMMERCIAL_AUDIO_PLUGINS` and :term:`COMMERCIAL_VIDEO_PLUGINS` statements
270(along with :term:`LICENSE_FLAGS_ACCEPTED`) includes the plugins or
271components into built images, thus adding support for media formats or
272components.
273
274.. note::
275
276 GStreamer "ugly" and "bad" plugins are actually available through
277 open source licenses. However, the "ugly" ones can be subject to software
278 patents in some countries, making it necessary to pay licensing fees
279 to distribute them. The "bad" ones are just deemed unreliable by the
280 GStreamer community and should therefore be used with care.
Andrew Geissler517393d2023-01-13 08:55:19 -0600281
282Maintaining Open Source License Compliance During Your Product's Lifecycle
283==========================================================================
284
285One of the concerns for a development organization using open source
286software is how to maintain compliance with various open source
287licensing during the lifecycle of the product. While this section does
288not provide legal advice or comprehensively cover all scenarios, it does
289present methods that you can use to assist you in meeting the compliance
290requirements during a software release.
291
292With hundreds of different open source licenses that the Yocto Project
293tracks, it is difficult to know the requirements of each and every
294license. However, the requirements of the major FLOSS licenses can begin
295to be covered by assuming that there are three main areas of concern:
296
297- Source code must be provided.
298
299- License text for the software must be provided.
300
301- Compilation scripts and modifications to the source code must be
302 provided.
303
304There are other requirements beyond the scope of these three and the
305methods described in this section (e.g. the mechanism through which
306source code is distributed).
307
Andrew Geissler220dafd2023-10-04 10:18:08 -0500308As different organizations have different ways of releasing software,
309there can be multiple ways of meeting license obligations. At
310least, we describe here two methods for achieving compliance:
311
312- The first method is to use OpenEmbedded's ability to provide
313 the source code, provide a list of licenses, as well as
314 compilation scripts and source code modifications.
315
316 The remainder of this section describes supported methods to meet
317 the previously mentioned three requirements.
318
319- The second method is to generate a *Software Bill of Materials*
320 (:term:`SBoM`), as described in the ":doc:`/dev-manual/sbom`" section.
321 Not only do you generate :term:`SPDX` output which can be used meet
322 license compliance requirements (except for sharing the build system
323 and layers sources for the time being), but this output also includes
324 component version and patch information which can be used
325 for vulnerability assessment.
326
327Whatever method you choose, prior to releasing images, sources,
328and the build system, you should audit all artifacts to ensure
329completeness.
Andrew Geissler517393d2023-01-13 08:55:19 -0600330
331.. note::
332
333 The Yocto Project generates a license manifest during image creation
Andrew Geissler220dafd2023-10-04 10:18:08 -0500334 that is located in
335 ``${DEPLOY_DIR}/licenses/<image-name>-<machine>.rootfs-<datestamp>/``
Andrew Geissler517393d2023-01-13 08:55:19 -0600336 to assist with any audits.
337
338Providing the Source Code
339-------------------------
340
341Compliance activities should begin before you generate the final image.
342The first thing you should look at is the requirement that tops the list
343for most compliance groups --- providing the source. The Yocto Project has
344a few ways of meeting this requirement.
345
346One of the easiest ways to meet this requirement is to provide the
347entire :term:`DL_DIR` used by the
348build. This method, however, has a few issues. The most obvious is the
349size of the directory since it includes all sources used in the build
350and not just the source used in the released image. It will include
351toolchain source, and other artifacts, which you would not generally
352release. However, the more serious issue for most companies is
353accidental release of proprietary software. The Yocto Project provides
354an :ref:`ref-classes-archiver` class to help avoid some of these concerns.
355
356Before you employ :term:`DL_DIR` or the :ref:`ref-classes-archiver` class, you
357need to decide how you choose to provide source. The source
358:ref:`ref-classes-archiver` class can generate tarballs and SRPMs and can
359create them with various levels of compliance in mind.
360
361One way of doing this (but certainly not the only way) is to release
362just the source as a tarball. You can do this by adding the following to
363the ``local.conf`` file found in the :term:`Build Directory`::
364
365 INHERIT += "archiver"
366 ARCHIVER_MODE[src] = "original"
367
368During the creation of your
369image, the source from all recipes that deploy packages to the image is
370placed within subdirectories of ``DEPLOY_DIR/sources`` based on the
371:term:`LICENSE` for each recipe.
372Releasing the entire directory enables you to comply with requirements
373concerning providing the unmodified source. It is important to note that
374the size of the directory can get large.
375
376A way to help mitigate the size issue is to only release tarballs for
377licenses that require the release of source. Let us assume you are only
378concerned with GPL code as identified by running the following script:
379
380.. code-block:: shell
381
382 # Script to archive a subset of packages matching specific license(s)
383 # Source and license files are copied into sub folders of package folder
384 # Must be run from build folder
385 #!/bin/bash
386 src_release_dir="source-release"
387 mkdir -p $src_release_dir
388 for a in tmp/deploy/sources/*; do
389 for d in $a/*; do
390 # Get package name from path
391 p=`basename $d`
392 p=${p%-*}
393 p=${p%-*}
394 # Only archive GPL packages (update *GPL* regex for your license check)
395 numfiles=`ls tmp/deploy/licenses/$p/*GPL* 2> /dev/null | wc -l`
396 if [ $numfiles -ge 1 ]; then
397 echo Archiving $p
398 mkdir -p $src_release_dir/$p/source
399 cp $d/* $src_release_dir/$p/source 2> /dev/null
400 mkdir -p $src_release_dir/$p/license
401 cp tmp/deploy/licenses/$p/* $src_release_dir/$p/license 2> /dev/null
402 fi
403 done
404 done
405
406At this point, you
407could create a tarball from the ``gpl_source_release`` directory and
408provide that to the end user. This method would be a step toward
409achieving compliance with section 3a of GPLv2 and with section 6 of
410GPLv3.
411
412Providing License Text
413----------------------
414
415One requirement that is often overlooked is inclusion of license text.
416This requirement also needs to be dealt with prior to generating the
417final image. Some licenses require the license text to accompany the
418binary. You can achieve this by adding the following to your
419``local.conf`` file::
420
421 COPY_LIC_MANIFEST = "1"
422 COPY_LIC_DIRS = "1"
423 LICENSE_CREATE_PACKAGE = "1"
424
425Adding these statements to the
426configuration file ensures that the licenses collected during package
427generation are included on your image.
428
429.. note::
430
431 Setting all three variables to "1" results in the image having two
432 copies of the same license file. One copy resides in
433 ``/usr/share/common-licenses`` and the other resides in
434 ``/usr/share/license``.
435
436 The reason for this behavior is because
437 :term:`COPY_LIC_DIRS` and
438 :term:`COPY_LIC_MANIFEST`
439 add a copy of the license when the image is built but do not offer a
440 path for adding licenses for newly installed packages to an image.
441 :term:`LICENSE_CREATE_PACKAGE`
442 adds a separate package and an upgrade path for adding licenses to an
443 image.
444
445As the source :ref:`ref-classes-archiver` class has already archived the
446original unmodified source that contains the license files, you would have
447already met the requirements for inclusion of the license information
448with source as defined by the GPL and other open source licenses.
449
450Providing Compilation Scripts and Source Code Modifications
451-----------------------------------------------------------
452
Andrew Geissler220dafd2023-10-04 10:18:08 -0500453At this point, we have addressed all we need prior to generating the
Andrew Geissler517393d2023-01-13 08:55:19 -0600454image. The next two requirements are addressed during the final
455packaging of the release.
456
457By releasing the version of the OpenEmbedded build system and the layers
458used during the build, you will be providing both compilation scripts
459and the source code modifications in one step.
460
461If the deployment team has a :ref:`overview-manual/concepts:bsp layer`
462and a distro layer, and those
463those layers are used to patch, compile, package, or modify (in any way)
464any open source software included in your released images, you might be
465required to release those layers under section 3 of GPLv2 or section 1
466of GPLv3. One way of doing that is with a clean checkout of the version
467of the Yocto Project and layers used during your build. Here is an
468example:
469
470.. code-block:: shell
471
472 # We built using the dunfell branch of the poky repo
473 $ git clone -b dunfell git://git.yoctoproject.org/poky
474 $ cd poky
475 # We built using the release_branch for our layers
476 $ git clone -b release_branch git://git.mycompany.com/meta-my-bsp-layer
477 $ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer
478 # clean up the .git repos
479 $ find . -name ".git" -type d -exec rm -rf {} \;
480
481One thing a development organization might want to consider for end-user
482convenience is to modify
483``meta-poky/conf/templates/default/bblayers.conf.sample`` to ensure that when
484the end user utilizes the released build system to build an image, the
485development organization's layers are included in the ``bblayers.conf`` file
486automatically::
487
488 # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
489 # changes incompatibly
490 POKY_BBLAYERS_CONF_VERSION = "2"
491
492 BBPATH = "${TOPDIR}"
493 BBFILES ?= ""
494
495 BBLAYERS ?= " \
496 ##OEROOT##/meta \
497 ##OEROOT##/meta-poky \
498 ##OEROOT##/meta-yocto-bsp \
499 ##OEROOT##/meta-mylayer \
500 "
501
502Creating and
503providing an archive of the :term:`Metadata`
504layers (recipes, configuration files, and so forth) enables you to meet
505your requirements to include the scripts to control compilation as well
506as any modifications to the original source.
507
508Compliance Limitations with Executables Built from Static Libraries
509-------------------------------------------------------------------
510
511When package A is added to an image via the :term:`RDEPENDS` or :term:`RRECOMMENDS`
512mechanisms as well as explicitly included in the image recipe with
513:term:`IMAGE_INSTALL`, and depends on a static linked library recipe B
514(``DEPENDS += "B"``), package B will neither appear in the generated license
515manifest nor in the generated source tarballs. This occurs as the
516:ref:`ref-classes-license` and :ref:`ref-classes-archiver` classes assume that
517only packages included via :term:`RDEPENDS` or :term:`RRECOMMENDS`
518end up in the image.
519
520As a result, potential obligations regarding license compliance for package B
521may not be met.
522
523The Yocto Project doesn't enable static libraries by default, in part because
524of this issue. Before a solution to this limitation is found, you need to
525keep in mind that if your root filesystem is built from static libraries,
526you will need to manually ensure that your deliveries are compliant
527with the licenses of these libraries.
528
529Copying Non Standard Licenses
530=============================
531
532Some packages, such as the linux-firmware package, have many licenses
533that are not in any way common. You can avoid adding a lot of these
534types of common license files, which are only applicable to a specific
535package, by using the
536:term:`NO_GENERIC_LICENSE`
537variable. Using this variable also avoids QA errors when you use a
538non-common, non-CLOSED license in a recipe.
539
540Here is an example that uses the ``LICENSE.Abilis.txt`` file as
541the license from the fetched source::
542
543 NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENSE.Abilis.txt"
544