Improving Versioning
Summary
- proposal to address issues with consistent versions of machines
- OSE's core business is actually producing documentation for machines available over a lifetime
- Good about the current situation
- proven technology, easy-to-use centralized wiki
- easy to use Google Presentations cloud application for build manuals
- Problems with the current situation
- wiki unstructured and overwhelming
- no real versioning in wiki and Google Presentations
- reliance on volatile, proprietary Google cloud services
- binary format for Google Presentations prevents proper versioning
- Requirements
- no major change in current workflow
- keep the wiki as central point
- do not rely on the wiki as a central point
- apply proper versioning reflecting severity of change
- true consolidation of machines with lifetime releases
- versioning supporting modularity
- no reliance on proprietary software/cloud services
- support textual format as much as possible for universal versioning
- same ease-of-use as Google Presentations
- automatic index for machines, modules, projects on the wiki
- Proposal
- adopt semantic versioning
- adopt Git with submodules
- create lifetime releases with Zenodo or IPFS
- automatically generate wiki pages, build manuals from Git repositories
- find a way to use a textual format as an easy-to-use replacement for Google Presentations
- Concrete steps
- Investigate the above in the Hamburg STEAM Camp
Introduction
Several projects in OSE become viable products, for example the D3D Universal and D3D Pro are being sold as kits. However, the nature of OSE is that many projects are in a high rate of development and versions may change much in a short amount of time. For example, the D3D Universal changed considerably from the summer of 2019 to January 2020 and possibly as a result the documentation on the wiki was not consistent.
On this page we will try to address some the issues. We will first elaborate on the context, then we will discuss what is good in the current situation. We will then define the problem and finally outline a set of ideas and requirement to improve on the status quo.
Context
Many machines are reaching a state that makes them a viable product. For others to be able to use them, we need proper documentation that is consistent over versions. The machines are highly modular which means that some version of machine A uses some version of project B. If B advances, it may no longer be a fit to A. As such, it is necessary to maintain bookkeeping about which version of a different component is used.
We could ask ourselves what OSE's core business is. An acceptable answer would be producing the machines of the GVCS. However, a more refined answer would be that OSE produces documentation to build the machines of the GVCS. To be able to repair them, it is necessary to understand it, hence the documentation.
An interesting feature of the OSE machines is that they should be usable over a lifetime. However, this would also require for the documentation to be available over a lifetime. It is not clear whether that is guaranteed currently.
Another question is then: What does it mean that the machine is available over a lifetime? Is the machine (rather its documentation) built now available in 20 years? Is the documentation of that specific version available? The machines built now are versioned using year and month. Or is only the current documentation available of the machine in the current state?
Qualities of the current situation
The current system in place has various good features. The OSE wiki has pages for each project where information and build manuals are stored. A wiki is proven method with a low entrance threshold for users. The wiki logs changes to the pages and people making the changes. Major benefits are:
- one central point of entry
- anyone can join
- ease of use
- logs activity with changes
- proven technology
Many of the build manuals are created in Google Presentations. This is a highly easy-to-use cloud application. Anyone can view the presentation and download it in various formats such as Microsoft Powerpoint, Libre Office, or PDF. It also supports templates to provide a fixed style and the application keeps track of versions although these versions are not available to the public as opposed to the wiki pages.
Problem definition
Although there are certainly good things to say about the current situation, unfortunately it is also possible to name problems. One of the problems of the wiki is that it contains lots of information in a more or less unstructered way. Navigation is reliant on links between pages, there is machine information, firmwares, "meta" topics such as this, etc. A common complaint is that users find it hard to find their way and get overwhelmed with the content.
Although a wiki keeps track of changes, it is not really meant for versioning. To the best of my knowledge, it is not impossible to mark a page as being in a state that should be consolidated, for a machine release for example.
A problem with the way machines are versioned is that the version number does not reflect how severe a change is. Does a minor change lead to a new version number, for example a new Z-sensor on a 3D-printer? Perhaps it does, it may get version 20.04, but if next month the layout of the axes change, it may get version 20.05, whereas it is a major change. And 3D-printer 20.05 may make use of an improved universal axes but which version? Do we consistently change the 3D-printer version if the universal axis version number changes? Managing these version numbers requires lots of discipline in the current situation.
The build manuals are drafted in Google Presentations but this has drawbacks. Firstly, editing requires a Google account. Secondly, the application is a proprietary cloud application from a major company that has so many resources that they can build an extremely high quality cloud application. However, this results in high degree of vendor lock-in. Google cloud services are highly volatile, see the discontinuation of Google Reader, Google +, and the number of video conferencing applications that may or may not be merged: Google Meet, Google DUO, Google Hangouts, Google chat, and all of these may or may not be integrated with Gmail. The point is that being reliant on such a large, volatile company where services may be canceled, terms of service may change is not a very "open" way of working.
More practical issues are the following: The internal format is not exposed to the user. The format is binary instead of textual which makes tracking changes more cumbersome and reliant on the used application.
Requirements
Based on the current qualities and problems we can define a set of requirements:
1) No major change in the status quo
The current way of working with the wiki has obvious benefits. Changing the way OSE works in a major way is likely to be unnecessary and a whole community has to make a big change.
2) Keep the wiki as a central point
The wiki is a proven technology with a low entrance threshold. It brings a community together in a central place. However, central points have drawbacks, certainly in an open-source project.
3) Do not rely on the wiki as a central point
The danger of a central point is that if the wiki goes offline, lots of very interesting information, a legacy, will be lost. It may be possible to protect for this.
4) Apply proper versioning
A good approach would be to apply versioning in such a way that a version change reflects the severity of the change.
5) Lifetime releases
True consolidation of versions in the form of releases (releases of the documentation) that will never be lost and available for eternity or at least for the lifetime of the machine.
6) Versioning supporting modularity
The GVCS machines are highly modular. There should be a proper system in place that can keep track of the various version numbers of the sub-releases.
7) No reliance on proprietary cloud services
For an open project as OSE, it would be preferable not to rely on proprietary software, also not on cloud services.
8) Support textual format as much as possible
Proper versioning works best with textual formats. For versioning in binary formats one needs support from the application that produces the binary format, whereas versioning on textual formats can be done with any tool. This means that versioning becomes universal.
9) Google Presentations ease-of-use
For documentation of the machines, the build manuals, we would like to have the same ease-of-use as Google Presentations offers.
10) Automatic indexing
For the machines, the modules that are part of the machines it would be helpful if there were some form of automatic indexing. Essentially, it would be good to have more structure on the wiki.