Project

General

Profile

RestoreEfsNvDataBackupFile » History » Version 9

Denis 'GNUtoo' Carikli, 07/04/2020 06:32 AM

1 1 Denis 'GNUtoo' Carikli
h1. RestoreEfsNvDataBackupFile
2
3
{{TOC}}
4
5 5 Denis 'GNUtoo' Carikli
h2. Warning
6
7
These instructions are experimental and have not been tested yet.
8
9 1 Denis 'GNUtoo' Carikli
h2. Introduction
10
11
Many devices supported by Replicant have a partition with the modem data (IMEI, etc) which is called EFS.
12
13
That partition is used by Replicant to enable the modem to store data, by making files inside that partition available to the modem through a network protocol between Replicant and the modem.
14
15 6 Denis 'GNUtoo' Carikli
As with every other computer system, files can get corrupted, especially if the power goes away suddenly, while data is being written in some of the files.
16 1 Denis 'GNUtoo' Carikli
17 6 Denis 'GNUtoo' Carikli
The advantage of an architecture like that is that Replicant has more control, and the modem (which runs nonfree software) has less control over the data it uses. That control can be used to fix some level of files corruption.
18 1 Denis 'GNUtoo' Carikli
19 6 Denis 'GNUtoo' Carikli
Files corruptions in that partition can easily happen with batteries that are in very bad conditions: very old batteries can still last several hours, but if the smartphones tries to consume too much power (mA / miliampers) at once, the battery can be unable to provide that much power at the same time, which results in the voltage dropping and the smartphone or the tablet being turned off suddenly, without having the time to properly save any data it may be writing.
20 1 Denis 'GNUtoo' Carikli
21
In addition to the issue described above, the modem data partition contains data that is crucial (like the IMEI) and we don't know (yet) how to reconstruct the files that are in that partition. And without them, the modem will not work. This is why doing a backup of the data that is in this partition is crucial.
22
23
So if you didn't do that backup yet, you can follow the instructions in the [[BackupTheEFS]] wiki page.
24
25
The [[BackupTheEFS]] page has information on which devices have a modem data partition (some devices supported by Replicant don't have a modem for instance).
26
27
h2. Warning
28
29
If you don't want your device to break, really make sure you have a backup of the modem data partition. The backup can be done with the instruction that are in the [[BackupTheEFS]] wiki page.
30
31 7 Denis 'GNUtoo' Carikli
If all the files in the modem data partition are destroyed for a reason or another, you won't be able to use the modem anymore.
32 1 Denis 'GNUtoo' Carikli
33 7 Denis 'GNUtoo' Carikli
This means that you will not be able to receive SMS, make calls, get 3G data connections, etc. anymore, and we don't know how to repair that (yet). 
34
35
If you have any information on the structure of the files inside that partition, it would be great to tell us about it (through a bugreport, on the mailing list, etc) so we could progress on that side. 
36
37
We're mostly interested in finding where are various information needed to reconstruct the files like the IMEI location and format, or other information needed to get the modem working again.
38 1 Denis 'GNUtoo' Carikli
39
h2. Restoring nv_data.bin
40
41
If only the nv_data.bin file is corrupted , you can easily restore it, without having to restore a full backup.
42
43
h3. Making a new backup
44
45
Before starting, it's a good idea to make a new backup of the modem data partition, and name it in a way that is self-explanatory. For instance you could put it in a directory named 'backup-with-corrupted-nvdata-bin' not to confuse it with other backups that do not contain any corrupted files. As repeated before, you can use the [[BackupTheEFS]] instructions to do a backup.
46
47
Making a new backup has several advantages:
48
* If you lost your old backups you have a new one
49
* If you have your old backups somewhere else (like on another computer), you don't risk erasing them by mistake
50
51
Make sure not to erase the old backups while making a new one.
52
53
At the time of writing, the [[BackupTheEFS]] contain the following command:
54
<pre>
55
REPLICANT_EFS_BACKUP_DIR=~/replicant_devices/0123456789abcdef/backup_efs
56
mkdir -p $REPLICANT_EFS_BACKUP_DIR && cd $REPLICANT_EFS_BACKUP_DIR
57
</pre>
58
59
This creates a directory named replicant_devices in your home, with inside a directory named 0123456789abcdef, with inside a directory named backup_efs.
60
Then it makes you go in that directory from the command line. This way the next commands will save the backups in that directory.
61
62
We can easily substitute that command with another one:
63
<pre>
64
REPLICANT_EFS_BACKUP_DIR=~/replicant_devices/backup-with-corrupted-nvdata-bin-of-fourth-jully-2020/backup_efs
65
mkdir -p $REPLICANT_EFS_BACKUP_DIR && cd $REPLICANT_EFS_BACKUP_DIR
66
</pre>
67
68
Here instead of creating the 0123456789abcdef directory, it will create one named backup-with-corrupted-nvdata-bin-of-fourth-jully-2020. We can change that as we need to make sure that the directory has a name that is self explanatory, not to confuse the half-corrupted backups with ones that are not corrupted at all.
69
70
Alternatively if the replicant_devices, 0123456789abcdef or backup_efs directories don't exist, you can also follow the [[BackupTheEFS]] and rename the directories as you wish once the backup is done. 
71
72
Or if you have an old backup, you could rename its directories to make sure you don't accidentally overwrite it.
73
74
h3. Setup ADB
75
76
The [[BackupTheEFS]] page has instructions to setup ADB. So if you just did a backup you should be good.
77
78
However if you already had an old backup somewhere and didn't do a new one, you will need to make sure that ADB is setup.
79
80
To do that, you can follow the [[ADB|setting up ADB on your computer]] instructions, so that you can access a root shell on your device.
81
82
*NOTE*: when prompted on your Replicant device, make sure that you check the box that says *Always allow from this computer* when you grant your computer USB debugging permissions. Otherwise, you will be unable to obtain root shell access on your Replicant device when you reboot it into the recovery OS to actually perform the EFS backup.
83
84
*NOTE*: for security reasons, you may want to [[BackupTheEFS#Revoke-USB-debugging-permissions|revoke these non-expiring permissions]] once the EFS backup is complete.
85
86
h3. Reboot into the recovery
87
88
You can reboot your device into the recovery by one of the following methods:
89
90
h4. Using ADB
91
92
Run the following on your computer:
93
94
<pre>
95
adb reboot recovery
96
</pre>
97
98
h4. Using your Replicant device's user interface
99
100
Do the following:
101
102
# Make sure that your device is booted under Replicant.
103
# Press the power button until a menu appears.
104
# Press *Reboot*.
105
# Select *Recovery* and then press *ok*.
106
107
h3. Ensure that your device's system partition is mounted
108
109
In order to get a root shell in the recovery, your devices system partition must be mounted.
110
111
On certain devices, "the system partition is already mounted":https://redmine.replicant.us/issues/1934, so you might already be able to get a root shell without mounting the system partition again.
112
113
In any case, it's still best to do the following to make sure that the system partition is mounted:
114
115
# Select *Advanced*.
116
# Select *Mount /system*.
117
# Press the back key to get back to the general menu.
118
119
h3. Copy and pasting commands
120
121
The next sections will have commands that you can copy and paste. To ensure that they work correctly it's best to:
122
* Type them in the same terminal window
123
* Copy/paste one line at the same time. For instance if you copy 10 lines and just paste then, in some cases you could have some issues.
124
125
Also to make sure that they are really executed, you can press the 'enter' key after having pasted them. 
126
127
Pressing 'enter' twice will not hurt as the command will still be executed only once.
128
129 3 Denis 'GNUtoo' Carikli
h3. How this works
130 1 Denis 'GNUtoo' Carikli
131
The nv_data.bin file seem to be the file that is corrupted the most, probably because it is more often written to than the other files.
132
133
On most devices it can be found at several locations:
134
* /efs/nv_data.bin
135
* /efs/.nv_data.bak
136
137
The second one is a backup file. Note the '.' in front of the nv_data.bak. That dot ('.') is part of the file name. So that file name is '.nv_data.bak'.
138
139
When a file name starts with a dot, many tools (like file managers, command line utilities like 'ls', etc) will not list that file by default.
140
141
This convention enables to 'hide' some files. In GNU/Linux, this is typically used to 'hide' the configuration files used by graphical programs. Without that, it wound't be very convenient as you would see hundreds of extra files in your home. And in GNU/Linux the graphical programs still need to store their configuration somewhere.
142
143
For instance if you want to list all the files in your home in GNU/Linux, including the hidden ones you can use the following command:
144
<pre>
145
ls -a ~
146
</pre>
147
148 3 Denis 'GNUtoo' Carikli
The '~' is a shortcut/alias for your home, and the '-a' switch tells ls to list all files (including the hidden ones). Note that it will also list '.' and '..' which you can ignore in this tutorial (they are special 'shortcuts' to the current directory and the parent directory).
149 1 Denis 'GNUtoo' Carikli
150
So here we need to replace 'nv_data.bin' with '.nv_data.bak'. And as '.nv_data.bak' is a hidden file, we will use the concept of hidden files later on.
151
152 3 Denis 'GNUtoo' Carikli
h4. Mount the modem data partition
153
154 1 Denis 'GNUtoo' Carikli
To do that we first need to mount the modem data partition (named efs) to a directory (here we will use /efs).
155
156
Your device should already be in the recovery, ready to accept commands through adb.
157
158
Not all the devices have their modem data partition at the same place. Because of that, the mount instructions are device specific.
159
160 2 Denis 'GNUtoo' Carikli
You can use the following instructions to mount the modem data partition, but make sure that you use the ones that corresponds to your device.
161 1 Denis 'GNUtoo' Carikli
162 3 Denis 'GNUtoo' Carikli
h5. Galaxy S 2 (I9100), Galaxy S 3 (I9300), Galaxy S 3 4G (I9305), Galaxy Note (N7000), or Galaxy Note 2 (N7100)
163 1 Denis 'GNUtoo' Carikli
164
<pre>
165
adb shell "mkdir /efs"
166
adb shell "mount /dev/block/platform/*/by-name/EFS /efs"
167
</pre>
168
169 3 Denis 'GNUtoo' Carikli
h5. Galaxy Nexus (I9250)
170 1 Denis 'GNUtoo' Carikli
171
<pre>
172
adb shell "mkdir /efs"
173
adb shell "mount /dev/block/platform/*/*/by-name/efs /efs"
174
</pre>
175
176 3 Denis 'GNUtoo' Carikli
h5. Galaxy Tab 2 7.0 (P3100)
177 1 Denis 'GNUtoo' Carikli
<pre>
178
adb shell "mkdir /efs"
179
adb shell "mount /dev/block/platform/*/*/by-name/EFS /efs"
180
</pre>
181
182 3 Denis 'GNUtoo' Carikli
h4. Actually replace nv_data.bin
183 1 Denis 'GNUtoo' Carikli
184
Now that the modem data partition is mounted to /efs, we can proceed to replace nv_data.bin.
185
186
187 3 Denis 'GNUtoo' Carikli
First we can list all the files in /efs to make sure everything is good so far:
188 1 Denis 'GNUtoo' Carikli
<pre>
189 3 Denis 'GNUtoo' Carikli
adb shell "ls -a /efs/"
190 1 Denis 'GNUtoo' Carikli
</pre>
191
192 3 Denis 'GNUtoo' Carikli
Here with a Galaxy SIII (GT-I9300), it gives the following output:
193
<pre>
194
.                .nv_data.bak         FactoryApp    h2k.dat     nv_data.bin.md5 
195
..               .nv_data.bak.md5     bluetooth     imei        wifi            
196
.nv_core.bak     .nv_state            drm           nv.log      wv.keys         
197
.nv_core.bak.md5 00000000.authtokcont gyro_cal_data nv_data.bin 
198
</pre>
199 1 Denis 'GNUtoo' Carikli
200 3 Denis 'GNUtoo' Carikli
So we can clearly see the nv_data.bin and .nv_data.bak
201 1 Denis 'GNUtoo' Carikli
202 3 Denis 'GNUtoo' Carikli
We will then proceed to replace /efs/nv_data.bin by /efs/.nv_data.bak. This can be done with the following command:
203
<pre>
204
adb shell "cp /efs/.nv_data.bak /efs/nv_data.bin"
205
</pre>
206 1 Denis 'GNUtoo' Carikli
207 3 Denis 'GNUtoo' Carikli
It's a good idea to verify that everything went fine. This can be done with the following command:
208
<pre>
209
adb shell "cmp /efs/.nv_data.bak /efs/nv_data.bin"
210
</pre>
211 1 Denis 'GNUtoo' Carikli
212 3 Denis 'GNUtoo' Carikli
If everything went fine, it shound't print anything.
213
214
If not, it will print an error similar to this one:
215
<pre>
216
/efs/.nv_data.bak /efs/nv_data.bin differ: char 1, line 1
217
</pre>
218
Note that the 'char 1, line 1' might differ.
219
220
So now we finally replaced nv_data.bin, however we need to do the exact same operation to replace nv_data.bin.md5 with .nv_data.bak.
221
222
The nv_data.bin.md5 file contains something called a "checksum":https://en.wikipedia.org/wiki/Checksum : it's a short sequence of numbers and letters that can be used to matematically verify if nv_data.bin wasn't accidentally corrupted by verifying if all the data in nv_data.bin match the checksum in nv_data.bin.md5
223
224
As we replaced nv_data.bin by the content of .nv_data.bak, we need to also replace nv_data.bin.md5 by the content of .nv_data.bak.md5, otherwise the modem will think that nv_data.bin is corrupted.
225
226
To do that we can use the following command:
227
<pre>
228
adb shell "cp /efs/.nv_data.bak.md5 /efs/nv_data.bin.md5"
229
</pre>
230
231
Then we can also verify like before that evertyhign went fine:
232
<pre>
233
adb shell "cp /efs/.nv_data.bak.md5 /efs/nv_data.bin.md5"
234
</pre>
235
236
If not, we might have an output that looks like that:
237
<pre>
238
/efs/.nv_data.bak.md5 /efs/nv_data.bin.md5 differ: char 1, line 1
239
</pre>
240
241 4 Denis 'GNUtoo' Carikli
h4. Umount / Unmount the modem data partition
242 3 Denis 'GNUtoo' Carikli
243 4 Denis 'GNUtoo' Carikli
We're now mostly done. We still need to properly umount the modem filesystem partition to prevent corruptions.
244 3 Denis 'GNUtoo' Carikli
245 4 Denis 'GNUtoo' Carikli
We can do it with the following command:
246 1 Denis 'GNUtoo' Carikli
<pre>
247
adb shell "umount /efs" 
248
adb shell "rmdir /efs" 
249
</pre>
250
251
h2. Reboot your device
252
253 4 Denis 'GNUtoo' Carikli
Once the the umount is done, you can finally reboot your device into Replicant by one of two ways.
254 1 Denis 'GNUtoo' Carikli
255
You can reboot by running the following command on your computer:
256
<pre>
257
adb reboot
258
</pre>
259
260
Or alternatively you can use the Replicant recovery graphical user interface to reboot by selecting *Reboot system now*.
261
262
h2. Revoke USB debugging permissions
263
264
If you are concerned about security, you may want to revoke the non-expiring USB debugging permissions that you granted to your computer earlier (which you may of course grant again at any time, as desired). This control is located at *Settings* > *Developer options* > *Revoke USB debugging authorizations*.
265
266
That's it! Your device's EFS partition is now backed up. Your device should be running Replicant normally again.
267 8 Denis 'GNUtoo' Carikli
268
h2. See also
269
270
If you are looking for more advanced information on the EFS partition, here are some links:
271 9 Denis 'GNUtoo' Carikli
* [[XMMBoot#Modem-partitions]]: This has research on the modem firmware partitions. The 'NV' partition is interesting as it probably contains a template information to recreates files like nv_data.bin
272
* We have a tool named "nv_data-md5":https://git.replicant.us/replicant/hardware_replicant_libsamsung-ipc/tree/tools/nv_data-md5.c that can compute the non-standard md5 checksums used for nv_data.bin. That tool is part of "libsamsung-ipc":https://git.replicant.us/replicant/hardware_replicant_libsamsung-ipc .
273 8 Denis 'GNUtoo' Carikli
* We have a bug report on the issue (#2025 :Enable to recreate the EFS partition completely from scratch) however we need more information to continue working on it. We're mostly interested in the offset and format of any information needed to transform a template nv_data.bin file into a working one, like the IMEI location, the modem firmware version if needed, etc.