[OZAPRS] Does APRS-IS duplicate discard work?

[Justin Albury]

A VERY simple fix.......  Everyone needs to update software to a stable current iGate package.

I'm just saying ;-)

Justin Albury
J Albury Communications
justin at jacomms.com
0417246791

-----Original Message-----
From: OZAPRS [mailto:ozaprs-bounces at aprs.net.au] On Behalf Of Owen Duffy
Sent: Thursday, 23 October 2014 9:22 AM
To: ozaprs at aprs.net.au
Subject: [OZAPRS] Does APRS-IS duplicate discard work?

Hi All,

I have asked that question of myself several times over recent weeks whilst I have been reconciling raw packet displays from arps.fi with equipment under test.

So far, it looks good.

Every instance where I have seen apparent duplicates has been explained when I dig deep enough.

The two most common causes:
1. nodes that delay packets for an excessive time; and 2. igate software that corrupts packets.

In the case of 1, substandard nodes might delay packets for up to half an hour in my experience. VK2YGV-5 commonly delays packets for 10 minutes or so, I have seen VK2UWP delay packets for half an hour or more.

APRSIS has a cache of recent packets (or hashes) which it compares to packets as they arrive, but that cache is for a smallish window that copes with normal delays as packets are bucket brigaded through three or four digipeaters (yes, some people do that - for example see VK2ZHE-8 yesterday with a path ofWIDE1-2,WIDE2-2), but it does not cope with 5 minute delays, much less half an hour.

So, these delayed packets that are not recognised as such are accepted as the position of the tracker AT THE TIME RECEIVED BY APRSIS, and so they corrupt mapping.

Case 2 is a spin of of the sad state of software and maintenance procedures used in the network. For example, the most common corrupter of my packets is VK2ZEN using decade old aprsd with known problems.
aprsd has a know defect that it converts non printable characters in packets to a space, a result of ignorance on the part of the developer who didn't do his research.

I have previously found (and mentioned) that avoiding MICE (which does use non-printable characters) circumvents that problem for me, but I have also discovered the ZEN corrupts packets that don't appear to have non-printable characters. The latter occurs for any packets that use CRLF at the end of record. ZEN works ok for records with no terminator or with LF, but, you guessed it, it figures that CR is a non-printable and converts it to a space... which is not very obvious when you look at the raw packet list but if you look at the page source, you will see the substituted character. Now APRSIS does NOT trim trailing blanks before doing a hash of the record, so it regards a packet with CRLF and one with <blank>LF at the end of the record as different and so the APRSIS dat feed is corrupted.

Now the latter effect is the probably explanation of why ZEN's APRSIS stats so a remarkably low duplicate rate given that so many packets have already been through the Sydney area with several igates. If you corrupt the record, it appears unique and your duplicate rate will be low.

Having a low duplicate rate might be interpreted as a sign of adding real value by igating 'new' traffic, but beware, it could also be a sign of record corruption. I have found that more than 10 digis corrupt records that I generate and have monitored to test the performance of aprx code changes I have made.

My conclusion is that the APRS network is really low quality, running old software with known problems and not monitored and maintained, and much as the core APRS-IS network might attempt to discard duplicates, it is frustrated by the low quality feed.

73
Owen





_______________________________________________
OZAPRS mailing list
OZAPRS at aprs.net.au
http://lists.aprs.net.au/mailman/listinfo/ozaprs