LGPL compliance testing

Hey guys,

here's an initial draft on how to check for LGPL compliance:

https://github.com/RIOT-OS/RIOT/wiki/LGPL-compliancy-guide

This is for showing that some proprietary code has been compiled and linked with a specific version of RIOT.

I wrote the wiki entry out of my head, so maybe I missed something, but the general method worked in prior testing. :wink:

Should we go and automate this, or am I missing something?

Kaspar

p.s.: This is in no way any conclusion to the license discussion, see it as purely technical please.

Is RIOT_VERSION the only thing that is automatically generated for each build? I mean, if even a build date is included somewhere the result will not be identical.

I like the idea, however.

Best regards,

Joakim Gebart Eistec AB www.eistec.se

Hi,

Is RIOT_VERSION the only thing that is automatically generated for each build?

Seems like...

I mean, if even a build date is included somewhere the result will not be identical.

I tried this on ARM (for the hello-world example) and the resulting files had a matching MD5.

Kaspar

Nice!

I'm too tired to really think through it, but how does this fit in when keeping parts more close to the hardware proprietary? Say I don't want to publish my device driver, and maybe I don't want to share my pin-mapping?

Cheers, Hauke

Hi,

nice, I will try today how it would be possible to replace and validate when someone change parts of say the core, not the whole core though.

Best regards, Martin

Hi,

I'm too tired to really think through it, but how does this fit in when keeping parts more close to the hardware proprietary? Say I don't want to publish my device driver, and maybe I don't want to share my pin-mapping?

Well, if your device driver compiles into object files, there's no difference to any "application" code. It doesn't matter if you recompile the rest of the code and relink. What pin mapping your driver uses doesn't matter, either.

If you have to change core RIOT's linkerscript or board defines, you are changing LGPLed code and have to share those changes to be LGPL compliant.

Kaspar

Hey,

Hi,

yes, I will provide a diff patch to apply hooks to a closed library inside of a core .c file, that must be applied in the build process of RIOT to use the shipped library. The patch and the hooks are of course open source and provided under LGPL.

So, building a specific RIOT version with the applied patch for the hooks, linked with the provided library should be equal with the binary (If I'm not confusing something)

Best regards, Martin

Hi,

So, building a specific RIOT version with the applied patch for the hooks, linked with the provided library should be equal with the binary (If I'm not confusing something)

It should...

Note that the current build system adds something to the RIOT_VERSION string ("-dirty" IIRC) if you modify the current git HEAD. Both applying a patch (without committing) or removing the source files of your application will trigger that. That's why I set RIOT_VERSION in the examples.

You might want to either commit your patch to the core locally and use the resulting git revision for building/validating, or live with the dirty RIOT version, which should also work (in theory).

Kaspar

Hi Kaspar, how about adding a line in the guide about this md5 check, as a concrete example to “prove” that the files are indeed identical? Then we could ask some FSF people or lawyers if md5 check is considered a valid proof :wink: Cheers Emmanuel

Hi,

Hi,

to "refuel" the disussion on license switching and working with LGPL and closed source I've opened #2357 [1].

Best regards, Martin

[1] https://github.com/RIOT-OS/RIOT/pull/2357

Hi Kaspar,

  thanks for the wiki entry. I would like to understand the different steps better. Following questions and comments:

  (1) Compiling with RIOT_VERSION: Including the RIOT_VERSION works as a trust anchor to identify the source code base. Assuming a single (valid) and trustworthy version, you wouldn't need this ingredient. Right?

  (2) MD5 hash: This is for convenient reasons. You could also do a bitwise comparison. Right?

  (3) Fine-grained object archives: Having separate object archives that reflect RIOT modules is the key idea of the approach. Two questions remain: (a) Which types of applications can someone create without affecting the "core" (i.e., other .a-files) of the OS. (b) What is the "core" of the OS?

  (4) Can you explain why your approach would not be possible in other OSes, e.g., Contiki?

Thanks   matthias

Hi,

   (1) Compiling with RIOT_VERSION: Including the RIOT_VERSION works as a trust anchor to identify the source code base. Assuming a single (valid) and trustworthy version, you wouldn't need this ingredient. Right?

Correct.

   (2) MD5 hash: This is for convenient reasons. You could also do a bitwise comparison. Right?

Correct. I assume that MD5-hashing is practically equivalent to bitwise comparison.

   (3) Fine-grained object archives: Having separate object archives that reflect RIOT modules is the key idea of the approach. Two questions remain: (a) Which types of applications can someone create without affecting the "core" (i.e., other .a-files) of the OS. (b) What is the "core" of the OS?

Actually, the *archives* don't really matter. The linker extracts all archives and only considers contained .o files for linking.

That the application objects are conveniently archived into one .a file that can be easily used for this license verification is just a by-product.

(a) Usually the "core" .a files are searched before the application .a. That way, even if a developer accidently overwrites symbols (e.g., functions or variables) that should have been supplied by RIOT, the linker would probably just ignore it. Have to try that.

So in theory, developers shouldn't be able to affect the system .a's files. And I can't think of anything that might do so by accident, when a developer *wants* to be LGPL compliant, just writes an application, maybe adds a proprietary driver but otherwise "behaves" regarding build system manipulation.

There are probably a million ways to sneak in modified RIOT code and still have a valid verification using our method here. So even if the verification passes, that alone is not proof that LGPL hasn't been violated.

(b) In this context, by "core" I mean all the RIOT code, build system, linker scripts etc., everything that is not "the application".

Maybe the git checkout or the unzipped distribution archive?

   (4) Can you explain why your approach would not be possible in other OSes, e.g., Contiki?

That would be great. :wink:

But this is possible with these OSs as well. RIOT's build system just made this really easy as all "aplicatin" code is already compiled in one .a, and RIOT happily builds if that .a is supplied pre-compiled without the code.

Kaspar

Hi Kaspar,

> (3) Fine-grained object archives: Having separate object archives that > reflect RIOT modules is the key idea of the approach. Two questions > remain: (a) Which types of applications can someone create without > affecting the "core" (i.e., other .a-files) of the OS. (b) What is the > "core" of the OS? Actually, the *archives* don't really matter. The linker extracts all archives and only considers contained .o files for linking.

That the application objects are conveniently archived into one .a file that can be easily used for this license verification is just a by-product.

  OK.

(a) Usually the "core" .a files are searched before the application .a. That way, even if a developer accidently overwrites symbols (e.g., functions or variables) that should have been supplied by RIOT, the linker would probably just ignore it. Have to try that.

So in theory, developers shouldn't be able to affect the system .a's files. And I can't think of anything that might do so by accident, when a developer *wants* to be LGPL compliant, just writes an application, maybe adds a proprietary driver but otherwise "behaves" regarding build system manipulation.

  I think that was a misunderstanding. I was more asking about the "realm" of applications. The counter-arguments we had in the LGPL discussion were not primarily "can we proof LGPL compliance" but more "can we build reasonable, proprietary IoT applications" without violating LGPL compliance.

There are probably a million ways to sneak in modified RIOT code and still have a valid verification using our method here. So even if the verification passes, that alone is not proof that LGPL hasn't been violated.

  From a legal perspective it then seems worthless.

  To be honest: I like the idea from an exercise point of view but I don't see how this approach can help us wrt the license discussion.

Cheers   matthias

Hi,

So in theory, developers shouldn't be able to affect the system .a's files. And I can't think of anything that might do so by accident, when a developer *wants* to be LGPL compliant, just writes an application, maybe adds a proprietary driver but otherwise "behaves" regarding build system manipulation.

   I think that was a misunderstanding. I was more asking about the "realm" of applications. The counter-arguments we had in the LGPL discussion were not primarily "can we proof LGPL compliance" but more "can we build reasonable, proprietary IoT applications" without violating LGPL compliance.

Well, yes we can. There's just no perfect way of proving an application is compliant.

Assuming RIOT is already ported to a specific device, writing any proprietary application on top of that, even adding drivers for proprietary hardware, is possible without violating LGPL.

A basic RIOT port (not including drivers for all components) for the target probably needs to be LGPLed, though.

Our "method" here offers a make target that makes it easy to distribute everything necessary to enable re-linking, thus complying with that part of the LGPL. For everyone wanting to stay LGPL-compliant, this is useful.

There are probably a million ways to sneak in modified RIOT code and still have a valid verification using our method here. So even if the verification passes, that alone is not proof that LGPL hasn't been violated.

   From a legal perspective it then seems worthless.

Probably right. But there is no technical way to prove any license compliancy while keeping some code closed.

OTOH, the benefit of modifying RIOT itself and concealing it has *no* commercially viable use case to me, while being a huge burden on the development itself compared to just sharing the modifications in the first place.

So anyone writing a proprietary application and distributing it like this method proposes very probably complies to LGPL.

Everyone else would just not mention RIOT and ship it's product breaking LGPL, as the criminal energy to be not license compliant is there and going through the pain of pretending to be compliant just doesn't make sense.

Also, if any project passes this verification and *is* compliant, one <5 minute possibly NDAed look at the actual proprietary code would prove that nothing is concealed, making the defense to any "you are not compliant" accusations extremely easy.

That works the other way, too. If someone distributes code in this method's manner but conceals dirty tricks in order to ship LGPL breaking code, a <5 minute look would uncover such tricks.

So while this doesn't give legal proof, it makes making a case for the actual "truth" (proving compliancy / proving non-compliancy) a lot easier.

Anything passing this verification, but breaking LGPL on purpose is easily spotted with one look at the actual code.

   To be honest: I like the idea from an exercise point of view but I don't see how this approach can help us wrt the license discussion.

I don't agree.

LGPL requires distribution of proprietary object files and everything else to enable re-linking the LGPLed code with the proprietary code. This make target makes exactly that very easy, removing most if not all developer time consuming aspects of LGPL compliancy.

IMHO this even more reduces the license discussion to monetarization attractiveness vs. freedom for users.

Kaspar

Hi Kaspar,

  the core technical argument that you gave is: Your approach simplifies compliance check. Simplification is nice but does not introduce additional new functions in principle. From this perspective I don't see how this approach allows us to do things that we have not been able to do before (except we can do it 'easier'). Would you agree on this?

  Please note that I explicitly do not discuss if the simplification is useful or not (actually, I think the simplification is questionable) because I don't think that this the point now.

Cheers   matthias

Hi everyone,

for me it is unclear what are the technical limitations of the approach proposed by Kaspar (I am not a specialist in static/dynamic linking issues).

Can we claim that this solution offers the same level of “comfort” than bearing with LGPL + closed source in other well-known contexts (for example: Linux + closed source), which we could refer to ?

If yes: we might have both a technical method and a powerful argument that is easy to grasp for most people.

If no: what do we have exactly?

Best,

Emmanuel

Hi Matthias,

the core technical argument that you gave is: Your approach simplifies compliance check. Simplification is nice but does not introduce additional new functions in principle. From this perspective I don't see how this approach allows us to do things that we have not been able to do before (except we can do it 'easier'). Would you agree on this?

Not sure what you mean.

This is not a technical tool for anything. Complying to LGPL is *easy*, and this makes it even easier.

Part of the license discussion was the question how developers can develop proprietary (closed source) applications / products using LGPLed RIOT.

LGPL puts some restrictions on doing that, it is a copyleft license.

Those restrictions require the vendor of such an application to provide a way to relink the proprietary code with a newer version of the used library (RIOT in this case).

So I created a make target which packs together everything needed to comply to that part of the license.

Anyone using RIOT for building proprietary application now has an easy method of providing the files needed for that part of LGPL compliancy, "the files" meaning compiled object files. Those that would have been shipped as part of a firmware file anyways. They don't expose more code than a full compiled binary, it's the same.

Compared to other systems, any proprietary binary using an LGPLed library (e.g., the GNU libc used on all major Linux distributions) implicitly ships compiled object files and allows relinking.

As RIOT uses static linking, the proprietary object files have to be saved before completing the static link in order to enable replacing the LGPLed part later. This is basically all this PR tries to make supertrivial.

IMHO this does make developing proprietary code using LPGLed RIOT comparable to writing any proprietary application for other systems like Linux, MacOS, Windows, ... with slight differences only because of the nature of embedded systems.

Kaspar

Hi,

Can we claim that this solution offers the same level of "comfort" than bearing with LGPL + closed source in other well-known contexts (for example: Linux + closed source), which we could refer to ?

This "solution" just makes the neccessary steps to comply to LGPL, which are technically trivial but hard-to-understand for anyone not into linking, dead easy.

IMHO, this enables developing proprietary applications for RIOT with the same level of comfort as, e.g., developing any proprietary application for Linux using LGPLed libraries (which probably means all as even the GNU C library used on all major Linux distributions is LGPL).

Kaspar