Project

General

Profile

the gsd4t gps chip on galaxy s2 - terminal software required

Added by robin p over 10 years ago

hi, i'm going to analyse the output from this gps chip, but am struggling to find a serial port terminal application which is free software. denis, what did you use to do the logging on GSD4t? or can anyone recommend any other software?

cheers,

robin


Replies (36)

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Paul Kocialkowski over 10 years ago

Hum, why would you need a serial port terminal application? The way to analyze the output from the GPS chips is through strace. If you want to implement communication with the chip, you should write some quick code in C that does the job…

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Matija Nalis almost 10 years ago

Any estimate how complicated would it be to make GPS work? I'm assuming low level driver is actually working, and we "only" have problem interpreting what output means?

I've make a quick look at gpsd sources and GSD4t wiki and it looks like there is some similarity in the protocol (0xa0, 0xa2 header indicative of SiRF binary protocol).
Paul, do you think it might be worthwhile to try fiddling with it? I'm not experienced with mobile phone hacking, but I can write some tests in C and parse strace...
How would you suggest one should start looking at it?
There might even be chance to force the GSD4t to NMEA mode by some initialization? That would solve a lot of headache. (I think I've seen some other chipsets have that feature)

I'm interested because I'm trying to free my Samsung Galaxy S2 as much as possible (well... I'd have to live with some nonfree firmwares I guess), but I do need working GPS (phone is primarily used for OpenStreetMap mapping; few calls/SMS and web browser are actually considered extra features :). I'm currently on Cynanogenmod 9, but I'd like to improve amount of free code by jumping to Replicant if there is reasonable chance that GPS can be made working. Although still not free, it would be better then current situation, and pave the road for the more free future.

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Paul Kocialkowski almost 10 years ago

Any estimate how complicated would it be to make GPS work? I'm assuming low level driver is actually working, and we "only" have problem interpreting what output means?

You're exactly right.

I've make a quick look at gpsd sources and GSD4t wiki and it looks like there is some similarity in the protocol (0xa0, 0xa2 header indicative of SiRF binary protocol).

That's right, but the rest of the sequence doesn't match SiRF in any known version.

Paul, do you think it might be worthwhile to try fiddling with it? I'm not experienced with mobile phone hacking, but I can write some tests in C and parse strace...
How would you suggest one should start looking at it?

Yes, I definitely think it's worth it. I have tried a couple times and failed, though I'm sure it can be done. If you know C and strace already, you should be fine (to be honest, I don't have much more skills than that). Reading the assembly of the gps.exynos4.so binary might help too.

First thing to do is to get CyanogenMod with a kernel built with ro.secure=0 and other things useful for debugging and then tracing the library using: https://gitorious.org/replicant/crespo-gps-utils (see in utils, I have yet to write a proper modules tracer).

I already have traces though, but you probably want to get your own.

About the wiki, I haven't populated it with all the info I have in hand currently. What I do know is that the proprietary binary is reading the bare protocol and then somehow converts it into the SiRF OSP (One Socket Protocol), which is what shows on the logs. However, it's very hard to figure out what sequences of OSP (which we can decode because there is documentation about it and I think the reference implementation's source code leaked) match which binary sequence out of the protocol.

Something I haven't tried yet is using a fake serial, something like socat and feeding a replay of a capture of the protocol slowly and seeing how it reacts step by step.
Running the proprietary binary inside gdb might help too, but I'm not used to doing that.

Note that I have already figured out the firmware upload procedure, even though I haven't implemented it yet, so don't worry about it and focus on the core protocol.

There might even be chance to force the GSD4t to NMEA mode by some initialization? That would solve a lot of headache. (I think I've seen some other chipsets have that feature)

I wish I knew how to do that, that would save a lot of effort. The truth is, the proprietary software's main purpose is to convert that binary protocol to NMEA, so if there was a way to directly get it out of the chip, they would probably have done so already, but there are still chances it's not the case. Anyway, I have no clue how to trigger that mode, if it exists. Note that we might need to use the binary protocol to have AGPS and faster fixes (if that's something we want: it might be bad if it sends the location to a 3rd party server, which it likely does).

I'm interested because I'm trying to free my Samsung Galaxy S2 as much as possible (well... I'd have to live with some nonfree firmwares I guess), but I do need working GPS (phone is primarily used for OpenStreetMap mapping; few calls/SMS and web browser are actually considered extra features :). I'm currently on Cynanogenmod 9, but I'd like to improve amount of free code by jumping to Replicant if there is reasonable chance that GPS can be made working. Although still not free, it would be better then current situation, and pave the road for the more free future.

I would really really love to be able to use the phone for OSM mapping as well. I used to do it with an Openmoko FreeRunner (working GPS) and now I'm rather taking lots and lots of photos and not doing any geographical tagging on the field.

Let me know what you think about all this.

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Matija Nalis almost 10 years ago

I've found out that instead of directly sniffing /dev/ttySAC1, one can set DEBUGGING_FILES=1 in /system/etc/sirfgps.conf
and get nice output - both raw A0 A2 sequences in /data/SLCLog.gp2, and some parts of decoding in progress in sirf_interface_log.txt, BriefLog.txt, DetailedLog.txt and AGPSLog.txt

As my only phone is production one which I cannot foobar (too much), I'm hoping I might get some useful info out of those.
It doesn't look like one to one mapping - DetailedLog mentions two OSP messages received per second, and there are more like 50 A0A2 binary sequences in that same second. Still, it at least looks hopefull, so I'll try to allocate some time to look more into it. I'll update as I find something...

If that doesn't give sufficient results, disassembly of /system/lib/hw/gps.exynos4.so might be in order, but it is probably overwhelming (1.7MB ELF file, over 1MB of that is .text section -- and I remember what HUGE amount of code were few tens of KB back in DOS days, and that was with excellent tools and ability to debug in realtime on the same platform, hardware breakpoints and all kinds of neat stuff; not to mention faster brain)

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Matija Nalis almost 10 years ago

Some progress info - SLCLog.gp2 which is generated on android with debug logs is actual processed data in SiRF binary format, which can be read (when converted to binary representation) by gpsd(8). I've managed to capture raw native data od GSD4t with strace, and have written handy tool to process that data in readable/hackable format.
Which helped me to find basic container structure for GSD4t raw data, which is quite different than SiRF binary.
It contains some strange sequence counters (two of them), size, CRC-16 for headers and CRC-16 for data...

Still nothing on actual packets content (or how to convert it), but it is a start.

Tools and docs gathered so far can be found at https://github.com/mnalis/gsd4t_parser

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by E3V3A XDA almost 10 years ago

@Matija: I've posted some links on your GitHUb. I think that you can Google your way to find the protocols used, for reversing or providing your own code. Cheers!

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Paul Kocialkowski over 9 years ago

I've found out that instead of directly sniffing /dev/ttySAC1, one can set DEBUGGING_FILES=1 in /system/etc/sirfgps.conf
and get nice output - both raw A0 A2 sequences in /data/SLCLog.gp2, and some parts of decoding in progress in sirf_interface_log.txt, BriefLog.txt, DetailedLog.txt and AGPSLog.txt

That's correct, I was aware of that as well, I just didn't take the time to document all that properly on the wiki. I'll try to do it soon.

It doesn't look like one to one mapping - DetailedLog mentions two OSP messages received per second, and there are more like 50 A0A2 binary sequences in that same second. Still, it at least looks hopefull, so I'll try to allocate some time to look more into it. I'll update as I find something...

To my understanding, the chip provides pseudorange measurements through a transport protocol that somewhat resembles SiRF, but is not. That data is used by gpsd to figure out the position and then generates SiRF OSP (One Socket Protocol) messages, which are not SiRF either but are possible to figure out (the source code of an official OSP implementation was leaked, as well as some technical documentation of the protocol). That's what you can see on the logs. Finally, gpsd will translate the OSP data to NMEA and send it over to the gps module, which probably uses that the provide all the relevant infos to the framework (it may also use OSP, but that's not very relevant).

If that doesn't give sufficient results, disassembly of /system/lib/hw/gps.exynos4.so might be in order

I have already looked a bit there and I think it can help figure out how things work together here, but I doubt we'll get any of the magic behind the interpretation of the raw chip data out of it.

Which helped me to find basic container structure for GSD4t raw data, which is quite different than SiRF binary.

That's a nice first step! I had already looked at it a bit but gave up due to inconsistent results.

I've posted some links on your GitHUb.

E3V3A XDA XDA: Care to post the links here as well? Thanks ;)

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by E3V3A XDA over 9 years ago

@Paul: Sure here's the stuff:

OSP manual:
http://www.mt-system.ru/sites/default/files/docs/documents/sim18%20module%20osp%20manual%20%28cs-129291-dc-8%29%5B1%5D.pdf

I also found another GitHub repo with some info on the SiRF binary mode (aka OSP).
https://github.com/tessel/hardware/blob/master/modules-overview.md

The A2235-H is based on CSR SiRFstarIV chipset, which supports a novel GPS fix mode and a variety of low power states.
The A2235-H supports two different data modes: NMEA and SiRF binary mode (also called OSP mode). Technical Machine's default APIs use the module in NMEA mode.

And this SiRFnavIV manual:
http://www.maestro-wireless.com/download-dp40

10.3 SiRFawareTM Support
SiRFawareTM is a low-power operating mode that seeks to maintain low uncertainty in position, time, and frequency, and to maintain valid current Ephemeris using either data collected from satellites in view or Extended Ephemeris methods. The SiRFawareTM mode is entered using the One Socket Protocol, an extension of the SiRF Binary Protocol. Please refer to the appropriate manual. In order to request a fix and to exit SiRFawareTM it is necessary to toggle the ON_OFF pin. Toggling is done by pulling the signal to HIGH for about 200ms.

Furthermore, I would suggest looking at the applications handling the data coming out of this device. Is it handled by any Java code?
What other binaries is using this data? If its java, it should be easy to see something...

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by E3V3A XDA over 9 years ago

Paul Kocialkowski wrote:

Finally, gpsd will translate the OSP data to NMEA and send it over to the gps module, which probably uses that the provide all the relevant infos to the framework (it may also use OSP, but that's not very relevant).

I don't understand this. (Is this really what you meant?) Why would GPS chip output data to gpsd just to have it translated to something else and then sent back to gps modeule again. Doesn't make sense, or I've misunderstood.

If that doesn't give sufficient results, disassembly of /system/lib/hw/gps.exynos4.so might be in order

Cab you provide us a copy for download?
BTW. Have you guys actually checked in the JB sources?
I found the following files there:

board-gps-bcm475x.c
board-gps-gsd4t.c
drm_agpsupport.c
garmin_gps.c
mach-c1-gps.c
midas-gps.c
board-gps.h

And in one of them it talk about being in a "Garmin USB packet format",
and that packets may be incomplete. (inconsistent?) And other stuff:

/* packet ids used in private layer */
#define PRIV_PKTID_SET_DEBUG    1
#define PRIV_PKTID_SET_MODE     2
#define PRIV_PKTID_INFO_REQ     3
#define PRIV_PKTID_INFO_RESP    4
#define PRIV_PKTID_RESET_REQ    5
#define PRIV_PKTID_SET_DEF_MODE 6

#define ETX     0x03
#define DLE     0x10
#define ACK     0x06
#define NAK     0x15
...
#define MODE_NATIVE          0
#define MODE_GARMIN_SERIAL   1

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Paul Kocialkowski over 9 years ago

Why would GPS chip output data to gpsd just to have it translated to something else and then sent back to gps modeule again. Doesn't make sense, or I've misunderstood.

We are misunderstanding each other due to terminology. I use module in the sense described in the Developer guide. I do not refer to the GPS chip (hardware) as the GPS module. Instead, the GPS module is the software that is in charge of communicating with the Android framework directly (gps.omap4.so for instance).

Cab you provide us a copy for download?

You can find those files on the CyanogenMod archives for i9100 or maguro (Galaxy Nexus). The GPS daemon should be the only binary called gpsd and the GPS module (not very interesting) is at /system/lib/hw/gps.*.so (not gps.goldfish.so or gps.default.so though).

BTW. Have you guys actually checked in the JB sources?

I did take a look at the kernel-side GPS implementation but mostly found UART transport and GPIO-related things, which are not very relevant to the protocol.

And in one of them it talk about being in a "Garmin USB packet format",

That's on the other hand very interesting! Which file in particular is that? Note that only the Galaxy S 2 (I9100) and Galaxy Nexus are known to use the GSD4t, so perhaps these are about another device (mainly, the BCM4751, which also uses an unknown binary protocol).

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by E3V3A XDA over 9 years ago

That's on the other hand very interesting! Which file in particular is that?

garmin-gps.c

Those were part of the Samsung JellyBean Kernel driver update 3.
I packed them up here for you, with duplicates of JB xx and Update 3.
http://d-h.st/OER

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Paul Kocialkowski over 9 years ago

Well that's a driver for Garmin GPS, which is not our case here, so that's not very relevant, unless by some incredible luck, it turns out the GSD4t is using that very same Garmin protocol, but I don't buy it.

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by E3V3A XDA over 9 years ago

Hmm, that's too bad.
I really don't know anything about GPS output, but would question why
they would put Garmin code in the JB update, if it wasn't relevant?
(Unless they wannabe able to connect Garmin's via Bluetooth.)
Anyway, I'd still be curious what those "MODE"s are. Maybe that
chip can do both?

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Paul Kocialkowski over 9 years ago

Obviously, Samsung didn't write the Garmin driver, so it is totally unrelated. Looking at the source, it's at ./drivers/usb/serial/garmin_gps.c so obviously, it comes from mainline.
The copyright line says " * Copyright (C) 2006-2011 Hermann Kneissel ".

What would lead you to think that this might have any connection with the GSD4t?

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by E3V3A XDA over 9 years ago

@Paul: Because those JB sources + update is specific for I9100.
I didn't make that clear. But although I know the SGS2 come in ~20
different HW versions, the GT-I9100 should be only one, but perhaps
I'm wrong here. I wasn't thinking so closely about this, I was just
searching the source tree for any GPS related items and that one looked
quite promising. And really, I don't see why Samsung would choose a GPS
device they need to write more/new drivers for, than normally necessary.
And why would the HW OEM of GSD4t, change a long time established protocol
without backward compatibility. None of that make sense. I'm just
speculating that this chip may provide additional modes that give
us other output. So why not try to sniff that device port and try
to replay whats coming from kernel to GPS chip and see if output
changes to something you know?

I got those sources a long time ago from Samsung, AFAICR,
so yes, I could be completely wrong making this connection.

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Paul Kocialkowski over 9 years ago

Because those JB sources + update is specific for I9100.

Well no. The Linux kernel is not specific to the I9100. In fact, it supports a pretty amazing number of other devices ;) During the process of kernel development for their device, Samsung just took the Android Linux kernel, which is itself based off the mainline kernel, and added parts to support the I9100 hardware. But everything that was in mainline is still there!

I was just searching the source tree for any GPS related items and that one looked quite promising

Well you must have not been aware of the fact that the Linux kernel source tree contains thousands of drivers and only a very very small part are relevant to the I9100.

And really, I don't see why Samsung would choose a GPS device they need to write more/new drivers for, than normally necessary.

Well you'd be surprised here. Samsung engineers love writing stuff from scratch even when 2 or 3 iterations of the driver for the same chip already exist.
Moreover, the kernel code related to the GSD4t still is only transport (generic UART for the SoC and GPIO handling).

And why would the HW OEM of GSD4t, change a long time established protocol without backward compatibility. None of that make sense. I'm just speculating that this chip may provide additional modes that give

Sorry but you're being delusional here. That's not how reality works. In facts, manufacturers love coming up with their own protocols for stuff and not document anything about them. And trust me when I'm saying that they don't give a damn about backward compatibility or anything already existing.

The chip provides pseudorange measurements so it ought to be fairly simple (looking at the size of the firmware, I doubt it implements multiple protocols).

So why not try to sniff that device port and try to replay whats coming from kernel to GPS chip and see if output changes to something you know?

If you're still talking about the Garmin thing, then that would be a pure waste of time and I don't advise anyone who has time to spend on this task to go in that direction.

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by E3V3A XDA over 9 years ago

^^ Thanks Paul for clarifying. I realize I'm not able to help in this matter,
and I'm sorry in case I have caused any unnecessary digression.

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Matija Nalis over 9 years ago

Paul Kocialkowski wrote:

It doesn't look like one to one mapping - DetailedLog mentions two OSP messages received per second, and there are more like 50 A0A2 binary sequences in that same second. Still, it at least looks hopefull, so I'll try to allocate some time to look more into it. I'll update as I find something...

To my understanding, the chip provides pseudorange measurements

When you say "pseudorange measurements", do you mean just raw satelite pseudo-distances (so I have to do trilateration in software on main CPU to get actual coordinates), or does it calculate and then send actual coordinates? (excuse my confusion, I have some basic idea how GPS system works, but I am at loss which part of hardware/software/firmware actually does what part of calculations)

through a transport protocol that somewhat resembles SiRF, but is not. That data is used by gpsd to figure out the position and then generates SiRF OSP (One Socket Protocol) messages, which are not SiRF either but are possible to figure out (the source code of an official OSP implementation was leaked, as well as some technical documentation of the protocol). That's what you can see on the logs.

when you say "SiRF OSP is not SiRF either", I'm confused. I assume there is only one protocol (Originaly called SiRF binary / SiRF SSB?) and then there was a newer revision (called SiRF One Socket Protocol / SiRF OSP) which is basically the same thing - it just adds few new codes and uses few previously unused bits, but it is fully backwards compatibile and you can live without the new additions? Is that correct, or am I missing something?

Finally, gpsd will translate the OSP data to NMEA and send it over to the gps module, which probably uses that the provide all the relevant infos to the framework (it may also use OSP, but that's not very relevant).

Uh, is that how it works in replicant or in original ROM? I only have Cyanogenmod 9, and it doesn't have gpsd(8) process at all, and it looks something like this (to the best I can discern):


GPS chip (pysical HW, and special closed-source firmware running on GPS HW chip)
| (using pysical GPIO pins)
v
Linux kernel driver
| (using kernel module on main CPU, which peeks and pokes GPIO pins)
v
/dev/ttySAC1 (kernel-userland bridge)
| (outputs new binary protocol, the one we want to decode)
v
/system/lib/hw/gps.exynos4.so (closed source! loaded as shared lib by system_server via dlopen(3)?)
| (decodes new binary protocol and passes it via function calls to system_server below)
v
/system/bin/system_server 
| (outputs "fine location" via android API to Android aplication below)
v
satstat.apk (or other Android GPS app)
| (human-readable output to screen)
v
actual human looking at human-readable sattelite coordinates

is this correct or what am I missing/getting wrong? I was under impression that unknown new binary protocol was interpreted and parsed by closed-source gps.exynos4.so, and that it was the thing I should reverse engineer. Now I'm not so sure after your explanations :)

If that doesn't give sufficient results, disassembly of /system/lib/hw/gps.exynos4.so might be in order

I have already looked a bit there and I think it can help figure out how things work together here, but I doubt we'll get any of the magic behind the interpretation of the raw chip data out of it.

Hm, if not in system/lib/hw/gps.exynos4.so, where is then interpretation of raw chip data happening then?
SOMETHING has to parse it and make sense of it before it sends coordinates to some Android user application? what does gps.exynos4.so do then?

Which helped me to find basic container structure for GSD4t raw data, which is quite different than SiRF binary.

That's a nice first step! I had already looked at it a bit but gave up due to inconsistent results.

I've actually matched quite a lot of new binary data to their counterpart SiRF OSP messages (see for example https://github.com/mnalis/gsd4t_logs/blob/master/strace6-rst-short/strace.log.human.txt), but I still miss the coordinates. Now if what you meant above by "chip provides pseudorange measurements (only)" is that there won't be the coordinates at all in that binary data, then I will have to find out from what parts to calculate it?

Sorry for the long message, but I'd much appreciate if you can spare some clarifications to point me in right direction,,,

TIA,
Matija

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Paul Kocialkowski over 9 years ago

When you say "pseudorange measurements", do you mean just raw satelite pseudo-distances (so I have to do trilateration in software on main CPU to get actual coordinates), or does it calculate and then send actual coordinates?

It does not send actual coordinates as NMEA would do. I do not know exactly what these pseudorange measurements are in details, since I haven't been able to understand exactly what the data we get out of the chip really is. Pseaudorange measurements is what the documentation about the chip mentions.

(excuse my confusion, I have some basic idea how GPS system works, but I am at loss which part of hardware/software/firmware actually does what part of calculations)

I am not any more advanced to tell you the truth. As far as I understand, the software part is in charge of calculating the position from whatever the gps chip actually sends.

when you say "SiRF OSP is not SiRF either", I'm confused. I assume there is only one protocol (Originaly called SiRF binary / SiRF SSB?) and then there was a newer revision (called SiRF One Socket Protocol / SiRF OSP) which is basically the same thing - it just adds few new codes and uses few previously unused bits, but it is fully backwards compatibile and you can live without the new additions? Is that correct, or am I missing something?

You're right there is, I suppose, one SiRF protocol and different revisions of it. The latest one is OSP. I simply meant that it differs from the SiRF protocol we are used to see and that is implemented in the free gpsd. However I'm not sure OSP is just a set of new messages, it might be something new that looks like the old one. Anyway, it's not so much the issue since we can more or less figure out the OSP messages and we're not going to have to deal with it when writing a free implementation.

Uh, is that how it works in replicant or in original ROM? I only have Cyanogenmod 9, and it doesn't have gpsd(8) process at all, and it looks something like this (to the best I can discern):

Just to make things clear: there is nothing for that chip in Replicant, so I'm always referring to how things work on CyanogenMod. And you're right, I got totally confused and mixed up info about the BCM4751, the other problematic GPS chip found on many Android devices. So in the case of the GSD4t, there is only gps.exynos4.so/gps.omap4.so which has the whole implementation.

is this correct or what am I missing/getting wrong? I was under impression that unknown new binary protocol was interpreted and parsed by closed-source gps.exynos4.so, and that it was the thing I should reverse engineer. Now I'm not so sure after your explanations :)

This is correct. The GPS module is actually loaded from hardware/libhardware/hardware.c, the GPS interface definitions are at hardware/libhardware/include/hardware/gps.h and the framework uses the hardware module interface from: frameworks/base/services/jni/com_android_server_location_GpsLocationProvider.cpp which is then used by Java code to make it available to the applications.

Hm, if not in system/lib/hw/gps.exynos4.so, where is then interpretation of raw chip data happening then?
SOMETHING has to parse it and make sense of it before it sends coordinates to some Android user application? what does gps.exynos4.so do then?

I'm not saying that's not where it is, because that's the case, but decompilation is usually not clear enough (unless you are really good at reading assembly) to figure out which mathematical operations are involved. I would recommend using some other methods of decompilation.

I've actually matched quite a lot of new binary data to their counterpart SiRF OSP messages (see for example https://github.com/mnalis/gsd4t_logs/blob/master/strace6-rst-short/strace.log.human.txt), but I still miss the coordinates.

That's good! I think I got as far as that point as well on my last try. There are some references about OSP that we can use to figure out the messages that show up on the logs. However the real important stuff is understanding what the binary protocol that the chip sends really means and how to make any use of that to calculate all the relevant stuff we need for the GPS implementation.

Now if what you meant above by "chip provides pseudorange measurements (only)" is that there won't be the coordinates at all in that binary data, then I will have to find out from what parts to calculate it?

Exactly, but OSP is not the binary data we want to be looking at. OSP is generated internally from the software, based on the pseudorange measurements that the chip provides.

Sorry for the long message, but I'd much appreciate if you can spare some clarifications to point me in right direction,,,

Sorry for not being clear, I must have been tired, drunk, or both.

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by E3V3A XDA over 9 years ago

Hi Guys,
A couple of observations. First, let's establish that the GSD4t does indeed output OSP messages, and that we can (?) assume that most of these have been correctly decoded by Matija. However, the OSP protocol also allow for encapsulating many other output formats, most notably the old NMEA. When NMEA is output, there is a default format-mode for providing certain items. This is clearly visible in the initial part of the readable logfile as:

NMEA message output: GGA, GLL, GSA, GSV, RMC, VTG, MSS

These can be further described as follows:

MEI ICD: 2.3
SiRF GPS SW Version SN4_4.1.1-B16_LPL_DBG 01/17/2012 017
-------------------------------------------------------------------------------
NMEA    Have?   Description
-------------------------------------------------------------------------------
GGA     *       Time, position and fix type data
GLL     *       Latitude, longitude, UTC time of position fix and status
GSA     *       GPS receiver operating mode, satellites used in the position solution, and DOP values
GSV     *       Number of GPS satellites in view satellite ID numbers, elevation, azimuth, & SNR values
MSS     *       Signal-to-noise ratio, signal strength, frequency, and bit rate from a radio-beacon receiver
RMC     *       Time, date, position, course and speed data
VTG     *       Course and speed information relative to the ground
ZDA             PPS timing message (synchronized to PPS)
150             OK to send message
151             GPS Data and Extended Ephemeris Mask
152             Extended Ephemeris Integrity
154             Extended Ephemeris ACK

Now, reading at the Manual (as I posted on matej GitHub issue) about these: we find that GGA,GSA,RMC are output every 1 second and GSV every 1-5 seconds, depending. These are output all in the same NMEA block. Thus matching to exact 1 second intervals for the "unknown" parts of the Matiej log, we see this fits more or less perfectly. So we can move on to the problem, which is how these have been coded, since we don't recognize them.

Here's a few:

                                   counter +1 --+
                                                |
16:21:45.446  8470:      84 70 00 00 00 00      09 E1 48 64     00 03 ...
16:21:46.446  847A:      84 7A 00 00 00 00      0A DA F1 10     00 03 ... 
16:21:47.457  8484:      84 84 00 00 00 00      0B D4 B7 02     00 03 ... 
16:21:48.456  848E:      84 8E 00 00 00 00      0C CE D0 DD     00 03 ... 
16:21:49.462  8498:      84 98 00 00 00 00      0D C8 8B 1A     00 03 ... 
                            |
                            +-- counter +10 dec

But before we go on, according to the manual, there are also other supported output protocols that can be set with OSP MID 135 (0c87) and include:

Set Protocol – Message ID 135 (0x87)
--------------------------------
Null = 0
SiRF Binary = 1
NMEA = 2
ASCII = 3
RTCM = 4
USER1 = 5 (note1)
SiRFLoc = 6
Statistic = 7

So anyway, we have to watch out for two additional things.
1) Maybe the encapsulated data is in another format like one of the above.
2) That we have to check the payload endianess, that depend on both the protocol used and whether its showing floating/double precision data.

Where multi-byte values are in the payload data, neither the alignment nor the byte order are defined as part of the transport although SiRF ayloads uses the big-endian order.
.
Multi-byte values are transmitted MSB first unless noted in the message tables. However, there are exceptions for floating-point and double-precision values.

Let's solve this!

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by E3V3A XDA over 9 years ago

While we're on the subject of extra fields and very interesting data, do have a look at this paper:
"DECODING OF SIRF BINARY PROTOCOL"
DOI: 10.2478/v10018-012-0005-y
http://www.researchgate.net/profile/Bartlomiej_Oszczak/publication/258488096_Decoding_of_SiRF_Binary_Protocol/file/e0b49533139d392130.pdf
This explains something about the protocols and pseudo-ranges...

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by E3V3A XDA over 9 years ago

Also, the unknown MID 51 (0x33) is:

6.41 Tracker Load Status Report - Message ID 51, Sub ID 6
This is message is sent by the SLC asynchronously whenever a tracker load starts or completes.

Why not contact CSR and simply ask them what is inside those unknowns?

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Matija Nalis over 9 years ago

Paul Kocialkowski wrote:

It does not send actual coordinates as NMEA would do. I do not know exactly what these pseudorange measurements are in details, since I haven't been able to understand exactly what the data we get out of the chip really is. Pseaudorange measurements is what the documentation about the chip mentions.

I've looked some more, and it seems that is correct. A GSD4t solution comprises a GSD4t hardware implementation with SiRFHost™ software running on a host CPU.. I assume gps.exynos4.so contains SiRFHost, SiRFInstantFix, SiRFLSM which parse raw chip data, do pseudorange solving and output results as OSP to next layer.

To do the pseudorange solving ourselves, we need to write program like this (his surounding pages are treasure trove too!). And to do that, it seems we need to extract from raw GPS chip data the following:

- GPS receiver time
- Iono coefficients (several of them)
- Pseudoranges for at least 4 SVs
- Ephemeris (and clock parameters) for at least 4 SVs

I've actually matched quite a lot of new binary data to their counterpart SiRF OSP messages (see for example https://github.com/mnalis/gsd4t_logs/blob/master/strace6-rst-short/strace.log.human.txt), but I still miss the coordinates.

That's good! I think I got as far as that point as well on my last try. There are some references about OSP that we can use to figure out the messages that show up on the logs. However the real important stuff is understanding what the binary protocol that the chip sends really means and how to make any use of that to calculate all the relevant stuff we need for the GPS implementation.

yes. Luckily, gpsd parses (and uses, or at least shows some human readable debug info) more than a few SiRF binary / OSP messages, so that will tell us what they mean and what structure there are in. And there are tons of links on gpsd site, some of which might explain the rest of the OSP messages. And when we've mapped raw chip binary to OSP, that means we'll also know what raw chip binaries are.

Now if what you meant above by "chip provides pseudorange measurements (only)" is that there won't be the coordinates at all in that binary data, then I will have to find out from what parts to calculate it?

Exactly, but OSP is not the binary data we want to be looking at. OSP is generated internally from the software, based on the pseudorange measurements that the chip provides.

Yes, of course -- I'm looking at raw data from chip, but am using OSP debug to help determine what raw data is. For example, if chip has a block of 100 bytes of unknown binary data, and I can see that 90 bytes of that are also in OSP/SiRF binary message x, and that OSP message x is "ephemeris data", than I know what I've found, and can document it in parser. (just an example). With any luck, end result will be parser for raw GPS chip data (for lack of better name, I've named it "raw4t" in code) which will explain enough of the data so it can be used for trilateration measurements.

Thanks for the clarifications, that helped point me in right directions!

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by Matija Nalis over 9 years ago

E3V3A XDA wrote:

A couple of observations. First, let's establish that the GSD4t does indeed output OSP messages, and that we can (?) assume that most of these have been correctly decoded by Matija. However, the OSP protocol also allow for encapsulating many other output formats, most notably the old NMEA.

To be precise, it does not encapsulate NMEA, it switches protocol to NMEA (if it really encapsulated, NMEA text would be inside "A0A2xxxxxB0B3" OSP binary packets.

However, if you look at latest updates in github issue, I've explained why 'GSD4t solution' which supports OSP/NMEA is not the same as 'GSD4t chip' which we're interested in. So we're looking only in data in strace.log.human.txt (all the others files are just to help decode that file)

Here's a few:
[... counter+1, counter+10 ...]

A-ha! that is a nice catch! This is exactly what we're interested in!
They really mostly do follow that, however at one point the "+1" counter becomes "+0" and at the same time "+10" counter becomes "+1". Interesting... sound to me like it is maybe some list, and if the packet is too big it gets cut into few smaller ones. I'll make a note and try to see more why that happens. Thanks!

1) Maybe the encapsulated data is in another format like one of the above.

maybe. If you can find some docs on those other formats (RTCM, User1, SiRFLoc, Statistic), it can be checked.

2) That we have to check the payload endianess, that depend on both the protocol used and whether its showing floating/double precision data.

yes, that is one important thing to watch out for.

While we're on the subject of extra fields and very interesting data, do have a look at this paper:
"DECODING OF SIRF BINARY PROTOCOL"
DOI: 10.2478/v10018-012-0005-y
This explains something about the protocols and pseudo-ranges...

Also very interesting, thanks! It seems we'll be needing to do pseudo-ranges solving ourselves, and this part is quite instructive! Do you know if software they wrote is also freely available somewhere?

Also, the unknown MID 51 (0x33) is:
6.41 Tracker Load Status Report - Message ID 51, Sub ID 6
This is message is sent by the SLC asynchronously whenever a tracker load starts or completes.

thanks, added. Guessed it must be something like that, as it occurs during init and shutdown only, so is not important for geting a GPS location. Still, the more information we can eliminate as useless, the less we have to look at.

Why not contact CSR and simply ask them what is inside those unknowns?

Because they might give that information under NDA (they usually do), and then you'd legally be banned for life on working on any open source project that might make use of such information. See http://en.wikipedia.org/wiki/Clean_room_design

RE: the gsd4t gps chip on galaxy s2 - terminal software required - Added by E3V3A XDA over 9 years ago

...sound to me like it is maybe some list, and if the packet is too big it gets cut into few smaller ones.

Yes, it depend on the direction. Data that goes IN to GSD4t are limited to 912 (0x390) bytes, while data coming OUT is 32767 bytes.
It's too late for me to look that up, but I read it earlier today.

Also, thanks to our Chinese friends, here is the datasheet for the GSD4t:

CS-106203-DSP2
http://wenku.baidu.com/view/875911caa1c7aa00b52acb29.html
(There are 2 other ones there as well, worth looking at.)

But I have not been able to download those yet, so be quick before the DMCA trolls get them closed down.

(1-25/36)