Models in below series start transmission of packet after receiving the
sequence of packets:
* Digidesign Digi00x family
* RME Fireface series
Additionally, models in Tascam FireWire series start multiplexing PCM
frames into packets enough after receiving packets. It's required to
transfer packets on-the-fly for the above models according to nominal
sampling transfer frequency before starting sequence replay.
This commit allows drivers to decide whether the engine transfers packet
on-the-fly or not.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210527122611.173711-4-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
ALSA IEC 61883-1/6 packet streaming engine uses pre-computed parameters
ideal for nominal sampling transfer frequency (STF) to transfer packets
to device since it was added 2011. As a result of user experience for a
decade, it is clear that the sequence is not suitable to some actual
devices. It takes the devices to generate noise, and causes any type of
discontinuity in the series of packet transferred from the device. It's
required for the engine to transfer packets according to effective STF.
The effective STF is given by media clock recovered by the sequence of
packet transferred from the target device. In the previous commit, the
sequence is already cached. The media clock recovery can be achieved by
analyzing the sequence.
In technological world, many ideas are proposed for media clock recovery.
However, the small part of them could be actually adopted in our case
since floating point arithmetic is not mostly available in Linux kernel
land.
This commit adopts the simple way from them; sequence replay, which means
that the sequence of parameters from incoming packet is used as is to
transfer outgoing packets. The media clock is not computed internally,
but the sequence of outgoing packet superficially looks to be generated by
the media clock.
The association between source and destination is decided when starting
AMDTP domain. When the target device supports a pair of isochronous packet
streams, the tx stream is source and the rx stream is destination. When it
supports two pair of streams, each of tx stream is associated to
corresponding rx stream in its order. When it supports less number of tx
streams than rx streams, the fist tx stream is selected for all of rx
streams. When it supports more tx streams than rx streams, the first tx
packet is associated to the rx stream.
As I noted in previous commit, the sequence of parameters from incoming
packet is different between devices, time to time. It is worse idea to
replay the sequence of parameters from a device for the sequence of
packet to the other devices even if they are in the same category of
device.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210527122611.173711-3-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
In design of audio and music unit in IEEE 1394 bus, feedback of
effective sampling transfer frequency (STF) is delivered by packets
transferred from device. The devices supported by ALSA firewire stack
are categorized to three groups regarding to it.
* Group 1:
* Echo Audio Fireworks board module
* Oxford Semiconductor OXFW971 ASIC
* Digidesign Digi00x family
* Tascam FireWire series
* RME Fireface series
* Group 2:
* BridgeCo. DM1000/DM1100/DM1500 ASICs for BeBoB solution
* TC Applied Technologies DICE ASICs
* Group 3:
* Mark of the Unicord FireWire series
In group 1, the effective STF is determined by the sequence of the number
of events per packet. In group 2, the sequence of presentation timestamp
expressed in syt field of CIP header is interpreted as well. In group 3,
the presentation timestamp is expressed in source packet header (SPH) of
each data block.
I note that some models doesn't take care of effective STF with large
internal buffer. It's reasonable to name it as group 0:
* Group 0
* Oxford Semiconductor OXFW970 ASIC
The effective STF is known to be slightly different from nominal STF for
all of devices, and to be different between the devices. Furthermore, the
effective STF is known to be shifted for long-period transmission. This
makes it hard for software to satisfy the effective STF when processing
packets to the device.
The effective STF is deterministic as a result of analyzing the batch of
packet transferred from the device. For the analysis, caching the sequence
of parameter in the packet is required.
This commit adds an option so that AMDTP domain structure takes AMDTP
stream structure to cache the sequence of parameters in packet transferred
from the device. The parameters are offset ticks of syt field against the
cycle to receive the packet and the number of data blocks per packet.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210527122611.173711-2-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
The member of callbacked in AMDTP stream structure is not used anymore.
Instead, ready_processing member is used to wake up yielding task of user
process.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210524031346.50539-3-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
The devices based on BeBoB ASICs or the devices in Tascam FireWire
series transfer a batch of NODATA packet or empty packet in the beginning
of packet streaming. To avoid processing them, current implementation uses
an option to skip processing content of tx packet during some initial
cycles. However, the hard-coded number is not enough useful.
This commit drops content of packets till the packet includes any event
firstly. The function of option is to skip processing content of tx packet
with any event after dropping.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210524031346.50539-2-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
In later commit, transfer delay is used in both IR and IT contexts. This
commit refactors regardless of transfer delay.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210522013303.49596-7-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Current implementation pools the sequence in AMDTP domain. This is convenient
regarding to memory usage and computation time, however inconvenient for the
devices such that several rx streams are expected to transfer timing
information independently.
This commit refactors to pool the sequence per rx packet stream.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210522013303.49596-3-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Many devices are unaware of syt field in rx CIP for playback timing.
This commit adds a flag to cancel processing syt field. Actually,
syt calculation is required to decide the number of events per rx packet.
The flag put 0xffff to CIP header of rx packet. On the other hand,
The value of syt field in CIP header of tx packet is unavailable. The
sequence of packet descriptor for tx packet includes 0 for the offset
of syt field to avoid computation.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210522013303.49596-2-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
When starting AMDTP domain, tasks in process context yields running CPU
till all of isochronous context get callback, with an assumption that
it's OK to process content of packet.
However several isochronous cycles are skipped to transfer rx packets, or
the content of rx packets are dropped, to manage the timing to start
processing the packets.
This commit changes the timing for tasks in process context to wake up
when processing content of packet is actually ready.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210520040154.80450-9-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
DICE ASICs support several pairs of isochronous packet streaming and
expect software to queue packets with the same timing information into
the same isochronous cycle.
This commit adds structure member to manage the cycle to start
processing packet in several IT contexts. The cycle is decided when
batch of isochronous cycle is skipped in callback to isochronous context
for IRQ target.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210520040154.80450-8-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Current implementation of ALSA IEC 61883-1/6 packet streaming engine
allows drivers to decide isochronous cycle to start IR context. This
option is mainly used to avoid processing the sequence of packet with
some quirks; e.g. discontinuity of counter. However, it's inconvenient
to fail to continue packet processing when the target device doesn't
start transmission of packet till the decided cycle.
This commit changes the behaviour. As an alternative to the start cycle
for IR context, the cycle count to drop content of packet in the beginning
of IR context.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210520040154.80450-6-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
DICE ASICs support several pairs of isochronous packet streaming. It's
convenient for drivers to process content of the packet in the same cycle
timing.
This commit adds structure member to manage the cycle to start
processing packet in several IR contexts. The cycle is decided in the
first callback of the IR contexts. The content of packet is dropped
till the cycle.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210520040154.80450-5-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Within devices supported by drivers in ALSA firewire stack, OXFW-based
devices and Fireface devices are known to skip isochronous cycle for
packet transmission. The former is due to the jumbo payload quirk. The
latter is due to vendor protocol in which empty packet is not
transferred in blocking mode.
Although nothing to do just for handling events of the packet, packet
continuity is necessarily for media clock recovery. This commit checks
whether any cycle is continue or not.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210518130048.146596-8-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
It brings some inconvenience in practice to use enumerated type for
variable to which bitwise OR with enumerator constant is assigned.
This commit replaces declarations of enumerated type with int type.
Suggested-by: Takashi Iwai <tiwai@suse.de>
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20210518084557.102681-3-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
The tasklet is an old API that should be deprecated, usually can be
converted to another decent API. In FireWire driver, a tasklet is
still used for offloading the AMDTP PCM stream handling. It can be
achieved gracefully with a work queued, too.
This patch replaces the tasklet usage in firewire-lib driver with a
simple work. The conversion is fairly straightforward but for the
in_interrupt() checks that are replaced with the check using the
current_work().
Note that in_interrupt() in amdtp_packet tracepoint is still kept as
is. This is the place that is probed by both softirq of 1394 OHCI and
a user task of a PCM application, and the work handling is already
filtered in amdtp_domain_stream_pcm_pointer().
Tested-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Acked-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20200909163659.21708-1-tiwai@suse.de
Signed-off-by: Takashi Iwai <tiwai@suse.de>
In previous commit, the sequence of syt offset and the number of data
blocks per packet is calculated for pool in AMDTP domain structure in
advance of processing outgoing packets.
This commit uses the sequence for outgoing packet processing to obsolete
per-stream processing of the sequence.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20200508043635.349339-11-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
In current implementation, sequence of syt offset and the number of data
blocks is generated when packets for outgoing stream are going to be
queued.
This commit generates and pools the sequence independently of the
processing of outgoing packets for future extension.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20200508043635.349339-10-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
For future extension, storage is required to store packet sequence in
incoming AMDTP stream to recover media clock for outgoing AMDTP stream.
This commit adds the storage to AMDTP domain for this purpose. The
packet sequence is represented by 'struct seq_desc' which has two
members; syt_offset and the number of data blocks. The size of storage
is decided according to the size of packet queue.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20200508043635.349339-9-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
In current implementation, AMDTP domain structure and AMDTP stream
structure has one way of reference from the former to the latter. For
future extension, bidirectional reference is needed.
This commit adds a member into stream structure to refer to domain
structure to which the stream belongs.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20200508043635.349339-4-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Some devices have a quirk to postpone transmission of isoc packet for
several dozen or hundred isoc cycles since configured to transmit.
Furthermore, some devices have a quirk to transmit isoc packet with
discontinued data of its header.
In 1394 OHCI specification, software allows to start isoc context with
certain isoc cycle. Linux firewire subsystem has kernel API to use it
as well.
This commit uses the functionality of 1394 OHCI controller to handle
the quirks. At present, this feature is convenient to ALSA bebob and
fireface driver. As a result, some devices can be safely handled, as
long as I know:
- MAudio FireWire solo
- MAudio ProFire Lightbridge
- MAudio FireWire 410
- Roland FA-66
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20191018061911.24909-7-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
This commit changes AMDTP domain to run on an IT context of 1394 OHCI as
IRQ target. No hardware interrupt is scheduled for the other isoc
contexts. All of the isoc context are processed in a callback for an isoc
context of IRQ target.
The IRQ target is automatically selected from a list of AMDTP streams,
thus users of AMDTP domain should add an AMDTP stream for IT context
at least.
The reason to select IT context as IRQ target is that the IT context
runs on local 1394 OHCI controller and it can be used as reliable,
constant IRQ generator. On the other hand, IR context can include skip
cycle according to isoc packet transferred by device.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20191018061911.24909-6-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
An isoc context for AMDTP stream is flushed to queue packet
by a call of pcm.ack. This commit extends this for AMDTP
domain.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20191018061911.24909-4-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
An isoc context for AMDTP stream is flushed to queue packet
by a call of pcm.pointer. This commit extends this for AMDTP
domain.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20191018061911.24909-3-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
This commit is a preparation to handle several IR/IT contexts in the same
domain by tasklet context for one of the IT context. Such IT context is
stored to AMDTP domain structure as 'irq_target'.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20191018061911.24909-2-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
ALSA IEC 61883-1/6 packet streaming engine controls 1394 OHCI controller
to generate hardware IRQ for fixed number of isochronous packets (=16)
since its first commit.
This commit allow the engine to generate it for variable period according
to the number of event to handle. For outgoing stream, internal
calculator is used to check the accumulated events. For incoming stream,
the number of data block in the packet of stream is used to check the
accumulated events. When it's unavailable, fixed number of packet
roughly calculated in advance is used instead of event counting.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20191017155424.885-11-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
The number of packets in packet buffer has been fixed number (=48) since
first commit of ALSA IEC 61883-1/6 packet streaming engine.
This commit allows the engine to use variable number of packets in the
buffer. The size is calculated by a parameter in AMDTP domain structure
surely to store the number of events in the packets of buffer. Although
the value of parameter is expected to come from 'period size' parameter
of PCM substream, at present 48 is still used.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20191017155424.885-2-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
In IEC 61883-6, it's called as 'event' what has presentation time
represented by timestamp in CIP header. Although the ratio of the number
of event against the number of data block is different depending on
event data type represented by the specific field in CIP header, it's
just one in the most cases supported by ALSA IEC 61883-1/6 engine.
In 1394 OHCI specification, applications can schedule hardware IRQ
by configuring descriptor with IRQ flag for packet against each
isochronous cycle. For future commit, I use the hardware IRQ for
isoc IT context to acknowledge the elapse of PCM period for both
playback/capture directions on AMDTP streams in the same domain.
This commit is a preparation for the above idea. This commit adds
a member into AMDTP domain structure to record the number of PCM frames.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Link: https://lore.kernel.org/r/20191007110532.30270-2-o-takashi@sakamocchi.jp
Signed-off-by: Takashi Iwai <tiwai@suse.de>
As a result to support AMDTP domain, no drivers call kernel APIs to
start/stop each AMDTP stream. This commit localize these APIs.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
This commit adds a kernel API to start a couple of isochronous contexts
for some AMDTP streams.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
This commit adds a kernel API to insert AMDTP stream to list in AMDTP
domain.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
This commit adds a kernel API to stop a couple of isochronous contexts
for AMDTP streams. The API is not protected with any lock primitive.
Callers should use this with enough lock against concurrent access.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
This commit adds 'struct amdtp_domain' structure. This structure
has list of instance of AMDTP stream to handle a couple of
isochronous contexts.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
This commit changes each of data block processing layer so that it
receives list of packet descriptor.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
This commit changes signature of callback function to call data block
processing layer with packet descriptor. At present, the layer is called
per packet.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
In current implementation of ALSA IEC 61883-1/6 packet streaming engine,
16 packets are handled in one interrupt of isochronous context of OHCI
1394.
Overall packet processing runs for each. However, this is not better in
a point to split the processing into several parts.
This commit is an attempt to add intermediate representation for
parameters required for the parts.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
In a previous commit, the variable passed from packet streaming layer
for syt variable is useless. This commit obsoletes it.
In my future work, the syt information is passed to data block processing
layer by another way.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Some protocols don't use syt field of CIP header to represent
presentation timestamp. For such protocol, ALSA IEC 61883-1/6
packet streaming engine uses a pointer into local variable for
the value of syt to call data block processing layer. However,
it can decide the value when initializing packet streaming
layer.
This commit adds 'syt_override' member for packet streaming
layer.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
This is a preparation for future commit that 'struct
amdtp_stream.data_block_count' does not represent the value of
data block count for current data block.
However, data block count is required for calculation of sequence index
in sequence-multiplied data channel. Some of data block processing layer
require it; e.g. for AM824 data block.
This commit passes data block count to the processing layer.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Recent firmware for Fireworks board module have a quirk to start
transmission of CIP with non-zero value for its data block counter.
In current implementation of ALSA firewire stack, the quirk is handled
by 'struct amdtp_stream.ctx_data.tx.first_dbc' with value 0x02. However,
the value comes from reverse engineering. It's better to handle this
quirk without the explicit value.
In a process to parse CIP header, the quirk of data block counter
affects decision of sequence index in sequence-multiplexed data channel;
i.e. MIDI conformant data channel. In Fireworks, the index is decided
by the number of data blocks from top of the same CIP, thus the value
of data block counter is useless.
This commit adds CIP_UNALIGHED_DBC flag and obsoletes the explicit
value for this quirk.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
The handlers for packet with CIP and without CIP include common codes.
This commit unifies them and remove an member for pointer to callback
function from data structure.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
In IR context, some quadlets of packet payload can be included into
context header. This is good for packet with CIP header because the
context payload buffer can includes data blocks only for with-CIP and
without-CIP pakets.
This commit uses 16 bytes IR context header for this purpose.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
The value of pointer to payload buffer is computed in each packet
handler, however the pointer can be decided before call of packet
handler.
This commit adds an argument for the pointer to the packet handler to
reduce codes to compute for the pointer.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
In current packet handler, the length of payload is given as an argument
of callback function, however this value is just required to process
payload of transferred isoc packet, thus just for IR context.
This commit replaces the argument for payload of packet with the
argument of context header. As a result, the length of payload is
computed in packet handler for IR context.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Some parameters of struct amdtp_stream is dependent on direction.
This commit uses union for such parameters to distinguish from
common parameters.
Signed-off-by: Takashi Sakamoto <o-takashi@sakamocchi.jp>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
