Project

General

Profile

RestoreEfsNvDataBackupFile » History » Version 12

Denis 'GNUtoo' Carikli, 07/04/2020 06:53 AM

1 1 Denis 'GNUtoo' Carikli
h1. RestoreEfsNvDataBackupFile
2
3
{{TOC}}
4
5 5 Denis 'GNUtoo' Carikli
h2. Warning
6
7 11 Denis 'GNUtoo' Carikli
These instructions are experimental and have not been tested yet, so it's better to have a backup of the modem partition before continuing.
8
9
Also, they were made for the Galaxy SIII (GT-I9300) and we need to make sure that it's supposed to work too on other devices by making sure that the filenames inside the modem data partition are the sames.
10 5 Denis 'GNUtoo' Carikli
11 12 Denis 'GNUtoo' Carikli
In case of doubt, you can still just read the instructions until the end and try to understand if they are good enough, before starting to follow them blindly.
12
13 1 Denis 'GNUtoo' Carikli
h2. Introduction
14
15
Many devices supported by Replicant have a partition with the modem data (IMEI, etc) which is called EFS.
16
17
That partition is used by Replicant to enable the modem to store data, by making files inside that partition available to the modem through a network protocol between Replicant and the modem.
18
19 6 Denis 'GNUtoo' Carikli
As with every other computer system, files can get corrupted, especially if the power goes away suddenly, while data is being written in some of the files.
20 1 Denis 'GNUtoo' Carikli
21 6 Denis 'GNUtoo' Carikli
The advantage of an architecture like that is that Replicant has more control, and the modem (which runs nonfree software) has less control over the data it uses. That control can be used to fix some level of files corruption.
22 1 Denis 'GNUtoo' Carikli
23 6 Denis 'GNUtoo' Carikli
Files corruptions in that partition can easily happen with batteries that are in very bad conditions: very old batteries can still last several hours, but if the smartphones tries to consume too much power (mA / miliampers) at once, the battery can be unable to provide that much power at the same time, which results in the voltage dropping and the smartphone or the tablet being turned off suddenly, without having the time to properly save any data it may be writing.
24 1 Denis 'GNUtoo' Carikli
25
In addition to the issue described above, the modem data partition contains data that is crucial (like the IMEI) and we don't know (yet) how to reconstruct the files that are in that partition. And without them, the modem will not work. This is why doing a backup of the data that is in this partition is crucial.
26
27
So if you didn't do that backup yet, you can follow the instructions in the [[BackupTheEFS]] wiki page.
28
29
The [[BackupTheEFS]] page has information on which devices have a modem data partition (some devices supported by Replicant don't have a modem for instance).
30
31
h2. Warning
32
33
If you don't want your device to break, really make sure you have a backup of the modem data partition. The backup can be done with the instruction that are in the [[BackupTheEFS]] wiki page.
34
35 7 Denis 'GNUtoo' Carikli
If all the files in the modem data partition are destroyed for a reason or another, you won't be able to use the modem anymore.
36 1 Denis 'GNUtoo' Carikli
37 7 Denis 'GNUtoo' Carikli
This means that you will not be able to receive SMS, make calls, get 3G data connections, etc. anymore, and we don't know how to repair that (yet). 
38
39
If you have any information on the structure of the files inside that partition, it would be great to tell us about it (through a bugreport, on the mailing list, etc) so we could progress on that side. 
40
41
We're mostly interested in finding where are various information needed to reconstruct the files like the IMEI location and format, or other information needed to get the modem working again.
42 1 Denis 'GNUtoo' Carikli
43
h2. Restoring nv_data.bin
44
45
If only the nv_data.bin file is corrupted , you can easily restore it, without having to restore a full backup.
46
47
h3. Making a new backup
48
49
Before starting, it's a good idea to make a new backup of the modem data partition, and name it in a way that is self-explanatory. For instance you could put it in a directory named 'backup-with-corrupted-nvdata-bin' not to confuse it with other backups that do not contain any corrupted files. As repeated before, you can use the [[BackupTheEFS]] instructions to do a backup.
50
51
Making a new backup has several advantages:
52
* If you lost your old backups you have a new one
53
* If you have your old backups somewhere else (like on another computer), you don't risk erasing them by mistake
54
55
Make sure not to erase the old backups while making a new one.
56
57
At the time of writing, the [[BackupTheEFS]] contain the following command:
58
<pre>
59
REPLICANT_EFS_BACKUP_DIR=~/replicant_devices/0123456789abcdef/backup_efs
60
mkdir -p $REPLICANT_EFS_BACKUP_DIR && cd $REPLICANT_EFS_BACKUP_DIR
61
</pre>
62
63
This creates a directory named replicant_devices in your home, with inside a directory named 0123456789abcdef, with inside a directory named backup_efs.
64
Then it makes you go in that directory from the command line. This way the next commands will save the backups in that directory.
65
66
We can easily substitute that command with another one:
67
<pre>
68
REPLICANT_EFS_BACKUP_DIR=~/replicant_devices/backup-with-corrupted-nvdata-bin-of-fourth-jully-2020/backup_efs
69
mkdir -p $REPLICANT_EFS_BACKUP_DIR && cd $REPLICANT_EFS_BACKUP_DIR
70
</pre>
71
72
Here instead of creating the 0123456789abcdef directory, it will create one named backup-with-corrupted-nvdata-bin-of-fourth-jully-2020. We can change that as we need to make sure that the directory has a name that is self explanatory, not to confuse the half-corrupted backups with ones that are not corrupted at all.
73
74
Alternatively if the replicant_devices, 0123456789abcdef or backup_efs directories don't exist, you can also follow the [[BackupTheEFS]] and rename the directories as you wish once the backup is done. 
75
76
Or if you have an old backup, you could rename its directories to make sure you don't accidentally overwrite it.
77
78
h3. Setup ADB
79
80
The [[BackupTheEFS]] page has instructions to setup ADB. So if you just did a backup you should be good.
81
82
However if you already had an old backup somewhere and didn't do a new one, you will need to make sure that ADB is setup.
83
84
To do that, you can follow the [[ADB|setting up ADB on your computer]] instructions, so that you can access a root shell on your device.
85
86
*NOTE*: when prompted on your Replicant device, make sure that you check the box that says *Always allow from this computer* when you grant your computer USB debugging permissions. Otherwise, you will be unable to obtain root shell access on your Replicant device when you reboot it into the recovery OS to actually perform the EFS backup.
87
88
*NOTE*: for security reasons, you may want to [[BackupTheEFS#Revoke-USB-debugging-permissions|revoke these non-expiring permissions]] once the EFS backup is complete.
89
90
h3. Reboot into the recovery
91
92
You can reboot your device into the recovery by one of the following methods:
93
94
h4. Using ADB
95
96
Run the following on your computer:
97
98
<pre>
99
adb reboot recovery
100
</pre>
101
102
h4. Using your Replicant device's user interface
103
104
Do the following:
105
106
# Make sure that your device is booted under Replicant.
107
# Press the power button until a menu appears.
108
# Press *Reboot*.
109
# Select *Recovery* and then press *ok*.
110
111
h3. Ensure that your device's system partition is mounted
112
113
In order to get a root shell in the recovery, your devices system partition must be mounted.
114
115
On certain devices, "the system partition is already mounted":https://redmine.replicant.us/issues/1934, so you might already be able to get a root shell without mounting the system partition again.
116
117
In any case, it's still best to do the following to make sure that the system partition is mounted:
118
119
# Select *Advanced*.
120
# Select *Mount /system*.
121
# Press the back key to get back to the general menu.
122
123
h3. Copy and pasting commands
124
125
The next sections will have commands that you can copy and paste. To ensure that they work correctly it's best to:
126
* Type them in the same terminal window
127
* Copy/paste one line at the same time. For instance if you copy 10 lines and just paste then, in some cases you could have some issues.
128
129
Also to make sure that they are really executed, you can press the 'enter' key after having pasted them. 
130
131
Pressing 'enter' twice will not hurt as the command will still be executed only once.
132
133 3 Denis 'GNUtoo' Carikli
h3. How this works
134 1 Denis 'GNUtoo' Carikli
135
The nv_data.bin file seem to be the file that is corrupted the most, probably because it is more often written to than the other files.
136
137
On most devices it can be found at several locations:
138
* /efs/nv_data.bin
139
* /efs/.nv_data.bak
140
141
The second one is a backup file. Note the '.' in front of the nv_data.bak. That dot ('.') is part of the file name. So that file name is '.nv_data.bak'.
142
143
When a file name starts with a dot, many tools (like file managers, command line utilities like 'ls', etc) will not list that file by default.
144
145
This convention enables to 'hide' some files. In GNU/Linux, this is typically used to 'hide' the configuration files used by graphical programs. Without that, it wound't be very convenient as you would see hundreds of extra files in your home. And in GNU/Linux the graphical programs still need to store their configuration somewhere.
146
147
For instance if you want to list all the files in your home in GNU/Linux, including the hidden ones you can use the following command:
148
<pre>
149
ls -a ~
150
</pre>
151
152 3 Denis 'GNUtoo' Carikli
The '~' is a shortcut/alias for your home, and the '-a' switch tells ls to list all files (including the hidden ones). Note that it will also list '.' and '..' which you can ignore in this tutorial (they are special 'shortcuts' to the current directory and the parent directory).
153 1 Denis 'GNUtoo' Carikli
154
So here we need to replace 'nv_data.bin' with '.nv_data.bak'. And as '.nv_data.bak' is a hidden file, we will use the concept of hidden files later on.
155
156 3 Denis 'GNUtoo' Carikli
h4. Mount the modem data partition
157
158 1 Denis 'GNUtoo' Carikli
To do that we first need to mount the modem data partition (named efs) to a directory (here we will use /efs).
159
160
Your device should already be in the recovery, ready to accept commands through adb.
161
162
Not all the devices have their modem data partition at the same place. Because of that, the mount instructions are device specific.
163
164 2 Denis 'GNUtoo' Carikli
You can use the following instructions to mount the modem data partition, but make sure that you use the ones that corresponds to your device.
165 1 Denis 'GNUtoo' Carikli
166 3 Denis 'GNUtoo' Carikli
h5. Galaxy S 2 (I9100), Galaxy S 3 (I9300), Galaxy S 3 4G (I9305), Galaxy Note (N7000), or Galaxy Note 2 (N7100)
167 1 Denis 'GNUtoo' Carikli
168
<pre>
169
adb shell "mkdir /efs"
170
adb shell "mount /dev/block/platform/*/by-name/EFS /efs"
171
</pre>
172
173 3 Denis 'GNUtoo' Carikli
h5. Galaxy Nexus (I9250)
174 1 Denis 'GNUtoo' Carikli
175
<pre>
176
adb shell "mkdir /efs"
177
adb shell "mount /dev/block/platform/*/*/by-name/efs /efs"
178
</pre>
179
180 3 Denis 'GNUtoo' Carikli
h5. Galaxy Tab 2 7.0 (P3100)
181 10 Denis 'GNUtoo' Carikli
182 1 Denis 'GNUtoo' Carikli
<pre>
183
adb shell "mkdir /efs"
184
adb shell "mount /dev/block/platform/*/*/by-name/EFS /efs"
185
</pre>
186
187 3 Denis 'GNUtoo' Carikli
h4. Actually replace nv_data.bin
188 1 Denis 'GNUtoo' Carikli
189
Now that the modem data partition is mounted to /efs, we can proceed to replace nv_data.bin.
190
191
192 3 Denis 'GNUtoo' Carikli
First we can list all the files in /efs to make sure everything is good so far:
193 1 Denis 'GNUtoo' Carikli
<pre>
194 3 Denis 'GNUtoo' Carikli
adb shell "ls -a /efs/"
195 1 Denis 'GNUtoo' Carikli
</pre>
196
197 3 Denis 'GNUtoo' Carikli
Here with a Galaxy SIII (GT-I9300), it gives the following output:
198
<pre>
199
.                .nv_data.bak         FactoryApp    h2k.dat     nv_data.bin.md5 
200
..               .nv_data.bak.md5     bluetooth     imei        wifi            
201
.nv_core.bak     .nv_state            drm           nv.log      wv.keys         
202
.nv_core.bak.md5 00000000.authtokcont gyro_cal_data nv_data.bin 
203
</pre>
204 1 Denis 'GNUtoo' Carikli
205 3 Denis 'GNUtoo' Carikli
So we can clearly see the nv_data.bin and .nv_data.bak
206 1 Denis 'GNUtoo' Carikli
207 3 Denis 'GNUtoo' Carikli
We will then proceed to replace /efs/nv_data.bin by /efs/.nv_data.bak. This can be done with the following command:
208
<pre>
209
adb shell "cp /efs/.nv_data.bak /efs/nv_data.bin"
210
</pre>
211 1 Denis 'GNUtoo' Carikli
212 3 Denis 'GNUtoo' Carikli
It's a good idea to verify that everything went fine. This can be done with the following command:
213
<pre>
214
adb shell "cmp /efs/.nv_data.bak /efs/nv_data.bin"
215
</pre>
216 1 Denis 'GNUtoo' Carikli
217 3 Denis 'GNUtoo' Carikli
If everything went fine, it shound't print anything.
218
219
If not, it will print an error similar to this one:
220
<pre>
221
/efs/.nv_data.bak /efs/nv_data.bin differ: char 1, line 1
222
</pre>
223
Note that the 'char 1, line 1' might differ.
224
225
So now we finally replaced nv_data.bin, however we need to do the exact same operation to replace nv_data.bin.md5 with .nv_data.bak.
226
227
The nv_data.bin.md5 file contains something called a "checksum":https://en.wikipedia.org/wiki/Checksum : it's a short sequence of numbers and letters that can be used to matematically verify if nv_data.bin wasn't accidentally corrupted by verifying if all the data in nv_data.bin match the checksum in nv_data.bin.md5
228
229
As we replaced nv_data.bin by the content of .nv_data.bak, we need to also replace nv_data.bin.md5 by the content of .nv_data.bak.md5, otherwise the modem will think that nv_data.bin is corrupted.
230
231
To do that we can use the following command:
232
<pre>
233
adb shell "cp /efs/.nv_data.bak.md5 /efs/nv_data.bin.md5"
234
</pre>
235
236
Then we can also verify like before that evertyhign went fine:
237
<pre>
238
adb shell "cp /efs/.nv_data.bak.md5 /efs/nv_data.bin.md5"
239
</pre>
240
241
If not, we might have an output that looks like that:
242
<pre>
243
/efs/.nv_data.bak.md5 /efs/nv_data.bin.md5 differ: char 1, line 1
244
</pre>
245
246 4 Denis 'GNUtoo' Carikli
h4. Umount / Unmount the modem data partition
247 3 Denis 'GNUtoo' Carikli
248 4 Denis 'GNUtoo' Carikli
We're now mostly done. We still need to properly umount the modem filesystem partition to prevent corruptions.
249 3 Denis 'GNUtoo' Carikli
250 4 Denis 'GNUtoo' Carikli
We can do it with the following command:
251 1 Denis 'GNUtoo' Carikli
<pre>
252
adb shell "umount /efs" 
253
adb shell "rmdir /efs" 
254
</pre>
255
256
h2. Reboot your device
257
258 4 Denis 'GNUtoo' Carikli
Once the the umount is done, you can finally reboot your device into Replicant by one of two ways.
259 1 Denis 'GNUtoo' Carikli
260
You can reboot by running the following command on your computer:
261
<pre>
262
adb reboot
263
</pre>
264
265
Or alternatively you can use the Replicant recovery graphical user interface to reboot by selecting *Reboot system now*.
266
267
h2. Revoke USB debugging permissions
268
269
If you are concerned about security, you may want to revoke the non-expiring USB debugging permissions that you granted to your computer earlier (which you may of course grant again at any time, as desired). This control is located at *Settings* > *Developer options* > *Revoke USB debugging authorizations*.
270
271
That's it! Your device's EFS partition is now backed up. Your device should be running Replicant normally again.
272 8 Denis 'GNUtoo' Carikli
273
h2. See also
274
275
If you are looking for more advanced information on the EFS partition, here are some links:
276 9 Denis 'GNUtoo' Carikli
* [[XMMBoot#Modem-partitions]]: This has research on the modem firmware partitions. The 'NV' partition is interesting as it probably contains a template information to recreates files like nv_data.bin
277
* We have a tool named "nv_data-md5":https://git.replicant.us/replicant/hardware_replicant_libsamsung-ipc/tree/tools/nv_data-md5.c that can compute the non-standard md5 checksums used for nv_data.bin. That tool is part of "libsamsung-ipc":https://git.replicant.us/replicant/hardware_replicant_libsamsung-ipc .
278 8 Denis 'GNUtoo' Carikli
* We have a bug report on the issue (#2025 :Enable to recreate the EFS partition completely from scratch) however we need more information to continue working on it. We're mostly interested in the offset and format of any information needed to transform a template nv_data.bin file into a working one, like the IMEI location, the modem firmware version if needed, etc.