Project

General

Profile

DeveloperGuide » History » Version 85

Denis 'GNUtoo' Carikli, 06/18/2019 01:22 PM
rework the string entry as it's not a decompilation tool

1 31 Paul Kocialkowski
h1. Developer guide
2 1 Paul Kocialkowski
3 15 Paul Kocialkowski
{{>toc}}
4
5 3 Paul Kocialkowski
h2. Prerequisites
6
7 8 Paul Kocialkowski
Developing on Replicant isn't much harder than developing on any other free software project as it doesn't require specific knowledge. In fact, you'll probably learn a lot along the way regarding how hardware works, how the Android system is composed, how the kernel works, etc, but you don't need to know all of this to start. However a basic set of skills is required, among which:
8 83 Denis 'GNUtoo' Carikli
* C language programming skills and the ability to understand other languages such as C++, Java and shell scripts.
9 82 Denis 'GNUtoo' Carikli
* Makefile skills (no need to know every detail about it, a general idea of how Makefiles work is enough). If you don't know how Makefiles work, Wikipedia has "an good article on make":https://en.wikipedia.org/wiki/Make_(software)#Makefile that has some examples Makefiles.
10
* Git skills (basically, how to commit changes, send them to our repos, dealing with branches without making a mess, etc). If you don't know git yet, you can find some documentation about Git at: https://git-scm.com/documentation to learn it.
11 3 Paul Kocialkowski
12
If you think you can cope with the requirements, then developing on Replicant should cause you no particular issue.
13
14 70 Wolfgang Wiedmeyer
The [[Index#Replicant-porting-guides|porting guides]] provide instructions for porting a new device to Replicant and also offer some tips for developing on Replicant.
15 3 Paul Kocialkowski
16 70 Wolfgang Wiedmeyer
Have a look at the [[Tasks]] page and feel free to [[Index#Contact|ask around]] for help to get started.
17 40 Wolfgang Wiedmeyer
18 70 Wolfgang Wiedmeyer
h3. Notes on writing free software replacements
19 59 Wolfgang Wiedmeyer
20 70 Wolfgang Wiedmeyer
Writing free software replacements for non-free components may require more skills depending on what you're trying to achieve, though there may be people with the adequate knowledge to help you and from whom you will likely learn a lot.
21 50 Wolfgang Wiedmeyer
22 40 Wolfgang Wiedmeyer
h2. Code hosting and submitting patches
23
24 75 Kurtis Hanna
Replicant's source code is hosted at "git.replicant.us":https://git.replicant.us/replicant. If you plan to regularly contribute to Replicant and if you don't yet have a code hosting provider that satisfies your needs, you are welcome to host your Replicant-related projects there under your own username. You only need to [[People#Active-developers|contact one of Replicant's developers]] and ask for an account. Please include in your request the name, username and Email address that should be used for creating your account. Your repos will then show up on the "contributor repos":https://git.replicant.us/contrib page.
25 40 Wolfgang Wiedmeyer
26 53 Wolfgang Wiedmeyer
Replicant currently doesn't accept merge requests. There are two ways to get your patches included: You can either send them to the "mailing list":https://lists.osuosl.org/mailman/listinfo/replicant or open an issue on the "issue tracker":/projects/replicant/issues and attach the patches to the issue. Replicant developers will then review your changes.
27 40 Wolfgang Wiedmeyer
28 75 Kurtis Hanna
See the Git documentation for "creating a patch":https://git-scm.com/docs/git-format-patch. Patches can be sent with "@git send-email@":https://git-scm.com/docs/git-send-email. If it's too much hassle for you to set up @git send-email@, sending the patches with your favorite mail client should be fine, too.
29 3 Paul Kocialkowski
30 81 Denis 'GNUtoo' Carikli
h2. Pushing patches
31
32
If you already have the ability to push patches into the main Replicant repositories, you still need to send your patches to the mailing list if they are to be applied on a Replicant version that is currently supported, or if they apply to the Replicant website.
33
34
You will then need to wait either for:
35
* One other developer to review the patch and add "per's":https://stallman.org/articles/genderless-pronouns.html Acknowledgement.
36
* One week if no other developer did review your patches.
37
38 71 Wolfgang Wiedmeyer
h2. Writing free software replacements
39 1 Paul Kocialkowski
40 71 Wolfgang Wiedmeyer
Here are some tips that may help you achieving a free software replacement for a specific component (some may be more or less relevant regarding the nature of what the component does):
41
* Look for interested people from other projects: LineageOS people are constantly fighting with non-free blobs and are sometimes happy to help replacing one.
42 1 Paul Kocialkowski
* Try to make the non-free binary as verbose as possible by enabling all the possible debug options on the config file or such.
43 85 Denis 'GNUtoo' Carikli
* You can use @string@ to try to find out the command line debug options, list of commands, etc.
44 71 Wolfgang Wiedmeyer
* Run the program in @strace@ and analyze the trace to understand what the program does.
45
* Add verbose debug prints in the concerned kernel driver (with @printk@ and show them via the @dmesg@ tool).
46
* Read the corresponding kernel driver: you can sometimes learn a lot by reading comments or headers.
47
* Collect data out of the kernel driver (via debug prints) and out of the non-free binary (via debug prints on the upper-layer).
48
* If there is a mathematical algorithm involved, force the values returned by the kernel to the non-free binary and analyze how it reacts, for instance with spreadsheet software.
49 1 Paul Kocialkowski
* If you're directly dealing with a hardware component, try to find a datasheet for the chip, it may hold precious details. In the best case, you may also be able to find a reference software implementation!
50 84 Denis 'GNUtoo' Carikli
* Make sure that the methods you use are legal where you live, and that the Replicant project is able to legally redistribute your work. In doubt, you could ask the Replicant developers in the mailing list and/or at the private contact address. The Replicant project has also access to legal advise through the FSF.
51 2 Paul Kocialkowski
52
h2. Upstreaming work
53
54 1 Paul Kocialkowski
It is generally a good idea to send some changes back to upstream, assuming that they will benefit from it as well.
55 2 Paul Kocialkowski
56 1 Paul Kocialkowski
When it is about the replacement of a non-free component present in the upstream systems, make sure that your replacement is reliable and complete.
57
Contact the interested developers on the upstream projects before attempting to send your replacement.
58
59
h3. LineageOS
60
61
The LineageOS team uses Gerrit to manage patch submissions. The process to get your patch included in LineageOS repos is explained on their wiki: "Gerrit":http://wiki.lineageos.org/usinggerrit-howto.html#submitting-to-gerrit
62 2 Paul Kocialkowski
63
You can also push directly using git using the following scheme (untested):
64
<pre>
65
git push ssh://<sshusername>@review.lineageos.org:29418/LineageOS/<projectname> HEAD:refs/for/<branchname>
66
</pre>
67 25 Paul Kocialkowski
68
h3. AOSP
69
70
The Android Open Source Project uses Gerrit to manage patch submissions. Some information about submitting patches to AOSP is available: https://source.android.com/source/submit-patches.html
71
72
You can push to AOSP's review using:
73 3 Paul Kocialkowski
<pre>
74
git push https://android-review.googlesource.com/platform/system/core HEAD:refs/for/master
75
</pre>
76 43 Wolfgang Wiedmeyer
77 71 Wolfgang Wiedmeyer
h2. Commonly-used terminology
78 3 Paul Kocialkowski
79 71 Wolfgang Wiedmeyer
In order to keep everything clear and consistent, we use the following terms with a precise meaning in mind:
80
* *Driver*: Software that is part of the *kernel* (builtin or as a module) and deals with communicating with the hardware
81
* *Hardware Abstraction Layer (HAL)*: Software that runs in *user-space* and deals with communicating with the hardware (usually through a kernel driver)
82
* *module*: Android HALs are also often called modules, so we may referrer to e.g. the @audio HAL@ as the @audio module@
83
* *blob*: Proprietary HAL
84
* *firmware*: Software that does not run on the main processor (the CPU) but rather in a separate chip (e.g. the Wi-Fi firmwares runs on the Wi-Fi chip)
85 1 Paul Kocialkowski
86
h2. Wiki guidelines
87
88
In order the keep the wiki simple and consistent, a few guidelines must be followed when editing.
89
90
Regarding the content:
91
* Only Replicant-specific topics should be covered by the wiki: there is no point in writing usage guides for generic Android aspects, such as the user interface.
92
* The content on each page should *only* be relevant to the latest Replicant version: make sure to update the content with newer Replicant versions.
93
* Substantial changes must be discussed before modifying the wiki.
94
* A comment should be added in the comment field at the bottom that shortly describes the change.
95 72 Wolfgang Wiedmeyer
* Make use of the wiki formatting possibilities: "quick reference":/help/en/wiki_syntax_textile.html and "detailed syntax overview":/help/en/wiki_syntax_detailed_textile.html
96 1 Paul Kocialkowski
  By using @[[Index]]@, it's possible to link to the start page of the wiki.
97 6 Paul Kocialkowski
98 3 Paul Kocialkowski
Regarding the writing style:
99
* Every page in the wiki has to be written in correct English, we do not aim to provide information in any other language.
100 17 Paul Kocialkowski
* Addressing readers directly should be avoided when possible: Instead, what is described should always be the subject of sentences.
101 4 Paul Kocialkowski
* Links to pages should be incorporated in text (Instructions to [[ToolsInstallation#ADB|install ADB]] shouldn't be: Instructions to install ADB: [[ToolsInstallation#ADB]]).
102 14 Paul Kocialkowski
103
Regarding the naming of pages:
104 34 Paul Kocialkowski
* Pages must be named in a consistent manner ([[NexusSI902xFirmwares]] is not to be called [[FirmwaresOnTheI902xNexusS]]).
105 1 Paul Kocialkowski
* Common prefixes and suffixes should be used for naming new pages ([[BuildDependenciesInstallation]] is not to be called [[InstallationOfBuildDependencies]]).
106 34 Paul Kocialkowski
* Page and section names should always be named using nouns, not verbs, either active or not ([[DeveloperGuide]] is not to be called [[DevelopingGuide]]).
107
* Page titles shouldn't contain space or any delimiter character (such as - or_) but use upper-case characters as word delimiters ([[BuildDependenciesInstallation]] is not to be called [[Build_dependencies_installation]]).
108
* Page titles *as shown on the page* should use lower-case when upper-case is not needed, even if the page name uses upper-case as word delimiters.
109
110 49 Wolfgang Wiedmeyer
Regarding the naming of devices:
111 53 Wolfgang Wiedmeyer
* Devices should be named after their model number and codename, without mention of the manufacturer.
112 49 Wolfgang Wiedmeyer
* Common device naming conventions should be followed consistently (the @Galaxy S 3 (I9300)@ is not to be called @Samsung S3 GT-I9300@ or @Galaxy S III@).
113 34 Paul Kocialkowski
114 71 Wolfgang Wiedmeyer
h2. Repositories
115 34 Paul Kocialkowski
116 71 Wolfgang Wiedmeyer
When working with Replicant repos, make sure to avoid breaking things. For instance, if you push a commit introducing a compilation error, it will break the whole build process.
117
It is better to create separate branches (that are not used by the official manifest branches) when your work is still in progress.
118
Creating branches that add debug infos on a particular topic is usually a good idea since it will save you time next time you want to debug the same component.
119
120 80 Denis 'GNUtoo' Carikli
See [[SourceCodeRepositories]] for more details about how to create or mirror repositories on Replicant server.
121 71 Wolfgang Wiedmeyer
122
h3. When creating a repository
123 78 Denis 'GNUtoo' Carikli
124 71 Wolfgang Wiedmeyer
In order to keep repo naming consistent, please name repositories by their name on the tree, replacing the @/@ by @_@.
125
For instance, when forking the LineageOS repo: @android_device_samsung_crespo@, rename it to @device_samsung_crespo@ on the Replicant repos.
126
This creates a more consistent way of naming repositories and makes it easier when pushing: just look at the location in the source tree and replace @/@ by @_@.
127
128
h3. When creating a branch
129
130
Official Replicant branches are named the following way:
131
* The @replicant-@ prefix
132
* The Replicant version
133
134
Such as: @replicant-2.3@ This should be used on the projects repositories as well as the manifest repository.
135
Any other branch should be considered as Work In Progress (WIP) and thus not be part of any official branch of the manifest.
136
137
There is although one exception, with the @master@ branch, that can be used by any project and be in any manifest given that the code held in the @master@ branch will work on any Replicant version.
138 16 Paul Kocialkowski
139 33 Paul Kocialkowski
h2. New images release
140 1 Paul Kocialkowski
141 33 Paul Kocialkowski
# Modify the changelog in the vendor files:
142
<pre>
143 54 Wolfgang Wiedmeyer
cd path/to/replicant-6.0/vendor/replicant
144 33 Paul Kocialkowski
edit CHANGELOG.mkdn
145
git add CHANGELOG.mkdn
146 54 Wolfgang Wiedmeyer
git commit -sS -m "Replicant 6.0 0001 images release"
147
git push git@git.replicant.us:replicant/vendor_replicant.git replicant-6.0
148 33 Paul Kocialkowski
</pre>
149 65 Wolfgang Wiedmeyer
# Increment the release in the "release scripts":https://git.replicant.us/replicant/release-scripts:
150 33 Paul Kocialkowski
<pre>
151
cd path/to/release-scripts
152 58 Wolfgang Wiedmeyer
edit releasevars.sh
153
git add releasevars.sh
154 1 Paul Kocialkowski
git commit -sS -m "Replicant 6.0 0001 images release"
155
git push git@git.replicant.us:replicant/release-scripts.git replicant-6.0
156 58 Wolfgang Wiedmeyer
</pre>
157 63 Wolfgang Wiedmeyer
# Tag all the repositories with the release tag script:
158
<pre>
159
path/to/release-scripts/releasetag.sh path/to/replicant-6.0
160
</pre>
161 58 Wolfgang Wiedmeyer
# In the manifest repo, merge the replicant-6.0-dev branch into the replicant-6.0 branch and increment the release in the manifest:
162
<pre>
163
cd path/to/manifest
164
git checkout replicant-6.0
165
git merge replicant-6.0-dev
166
edit default.xml
167
git add default.xml
168
git commit -sS -m "Replicant 6.0 0001 images release"
169
git push git@git.replicant.us:replicant/manifest.git replicant-6.0
170 33 Paul Kocialkowski
</pre>
171
# Tag the manifest:
172 61 Wolfgang Wiedmeyer
<pre>
173
git tag -u 5816A24C10757FC4 replicant-6.0-0001 -m "Replicant 6.0 0001 images release"
174
git push git@git.replicant.us:replicant/manifest.git replicant-6.0-0001
175
</pre>
176 64 Wolfgang Wiedmeyer
# Verify all tags:
177
<pre>
178
cd .repo/manifests
179
git verify-tag $(git describe)
180
cd ../..
181 68 Wolfgang Wiedmeyer
repo forall -ec ' { echo "Verifying $REPO_PROJECT" && git verify-tag $(git describe) 2>/dev/null; } || { echo "Error: verification failed!" && exit 1; } '
182 64 Wolfgang Wiedmeyer
</pre>
183 56 Wolfgang Wiedmeyer
# Update prebuilts and start the build (in a newly opened shell with the Replicant keys and certificates installed):
184
<pre>
185
path/to/release-scripts/releasebuild.sh path/to/replicant-6.0
186
</pre>
187 33 Paul Kocialkowski
# Release the images with the release script:
188
<pre>
189 55 Wolfgang Wiedmeyer
rm -rf path/to/images/replicant-6.0/0001
190 54 Wolfgang Wiedmeyer
mkdir -p path/to/images/replicant-6.0/0001
191
path/to/release-scripts/release.sh path/to/replicant-6.0 path/to/images/replicant-6.0/0001
192 1 Paul Kocialkowski
</pre>
193 33 Paul Kocialkowski
# Sign the binaries with the release script:
194
<pre>
195 54 Wolfgang Wiedmeyer
path/to/release-scripts/release.sh path/to/replicant-6.0 path/to/images/replicant-6.0/0001 signatures
196 1 Paul Kocialkowski
</pre>
197 33 Paul Kocialkowski
# Compress the release files
198
<pre>
199 54 Wolfgang Wiedmeyer
cd path/to/images/replicant-6.0
200
tar -cjf 0001.tar.bz2 0001
201 1 Paul Kocialkowski
</pre>
202 33 Paul Kocialkowski
# Upload the release to OSUOSL:
203
<pre>
204 66 Wolfgang Wiedmeyer
rsync -P -4 -ze ssh 0001.tar.bz2 replicant@ftp-osl.osuosl.org:/home/replicant/data/images/replicant-6.0/
205 54 Wolfgang Wiedmeyer
</pre>
206 67 Wolfgang Wiedmeyer
# Unpack the release on OSUOSL, ensure permissions are correct and run the @trigger-replicant@ script
207 33 Paul Kocialkowski
# Update [[ReplicantImages]] with the release
208
# Update each device's page with the release
209
# Update [[ReplicantStatus]] with the latest status
210 62 Wolfgang Wiedmeyer
# Verify if other wiki pages need to be updated due to changes introduced by the release (e.g. [[Index#Replicant-build|build pages]] or [[ToolsInstallation]])
211 33 Paul Kocialkowski
# Announce the release on the blog
212
# Update the release on the website and IRC topic
213 5 Paul Kocialkowski
214 33 Paul Kocialkowski
h2. New device documentation
215 22 Paul Kocialkowski
216 19 Paul Kocialkowski
1. Create the device main page, following the naming guidelines applied to other devices (e.g. the Samsung Galaxy S II GT-I9100 is called *Galaxy S 2 (I9100)* and its page is [[GalaxyS2I9100]])
217 74 Wolfgang Wiedmeyer
2. Create all the related sub-pages (build guide, install guide and firmwares list at least), following the naming guidelines applied to other devices (e.g. [[GalaxyS2I9100Build]], [[GalaxyS2I9100Installation]] and [[GalaxyS2I9100LoadedFirmwares]])
218 5 Paul Kocialkowski
3. Link the sub-pages to the main page in the index
219
4. Update the [[ReplicantStatus]] page of the wiki with the current status of the device
220 47 Wolfgang Wiedmeyer
5. Modify the [[Index]] page of the wiki and add the new device in the following sections:
221
* [[Index#Replicant-status|Replicant status]]
222
* [[Index#Replicant-installation|Replicant installation]]
223
* [[Index#Replicant-build|Replicant build]]
224
* [[Index#Supported-devices|Supported devices]]
225 24 Paul Kocialkowski
226 43 Wolfgang Wiedmeyer
6. Add new issues categories to the Replicant project Redmine
227 73 Wolfgang Wiedmeyer
228
7. Add the device to the "Supported devices page":https://www.replicant.us/supported-devices.php on the website