| Package | Status |
|---|---|
| Ais.Net |  |
| Ais.Net.Models |  |
| Ais.Net.Receiver |  |
| Ais.Net.Receiver.Storage.Azure.Blob |  |
The AIS.NET project contains a series of layers, from a low-level high performance NMEA AIS sentence decoder, to a rich high-level C# 9.0 models of AIS message types, a receiver component that can listen to TCP streams of NMEA sentences and expose them as an IObservable<string> of raw sentences or an decoded IObservable<IAisMessage>, and finally a Storage Client implementation to persisting the raw NMEA sentence stream to Azure Blob storage for future processing.
A simple .NET AIS Receiver for capturing the Norwegian Coastal Administration's marine Automatic Identification System (AIS) AIVDM/AIVDO NMEA message network data (available under Norwegian license for public data (NLOD)) and persisting in Microsoft Azure Blob Storage.
The Norwegian Coastal Administration provide a TCP endpoint (153.44.253.27:5631) for broadcasting their raw AIS AIVDM/AIVDO sentences, captured by over 50 base stations, and covers the area 40-60 nautical miles from the Norwegian coastline.
This project contains a NmeaReceiver which consumes the raw NetworkStream, a NmeaToAisMessageTypeProcessor, which can decode the raw sentences into IAisMessage, and ReceiverHost which manages the process and exposes an IObservable<string> for raw sentences and an IObservable<IAisMessage> for decoded messages. ReceiverHost can be hosted in a console application or other runtime environments like .NET Interactive.
The project also includes a demo console which shows how the various pieces can fit together, including subscribing to the IObservable<string> and IObservable<IAisMessage> streams and displaying the results or batch the AIVDM/AIVDO sentences and write them to Azure Blob Storage using the Append Blob feature, to create timestamped hour-long rolling logs.
The purpose of this application is to provide sample data for Ais.Net - the .NET Standard, high performance, zero allocation AIS decoder. The majority of raw AIS data is only available via commercial sources, and thus creating AIS datasets large enough to test / benchmark Ais.Net is almost impossible.
The Norwegian Coastal Administration TCP endpoint produces:
- ~2.9 KB per second
- ~10.3 MB per hour
- ~248 MB per day
- ~1.7 GB per week
- ~7 GB per month
- ~81.4 GB per year
The AIS data is stored using the following taxonomy
<USER DEFINED CONTAINER NAME>/raw/yyyy/MM/dd/yyyyMMddTHH.nm4
An example directory listing, with a user defined container name of nmea-ais would look as follows:
\---nmea-ais
\---raw
\---2021
\---07
+---12
| 20210712T00.nm4 |
| 20210712T01.mm4 |
| 20210712T02.nm4 |
| 20210712T03.nm4 |
| 20210712T04.nm4 |
| 20210712T05.nm4 |
| 20210712T06.nm4 |
| 20210712T07.nm4 |
| 20210712T08.nm4 |
| 20210712T09.nm4 |
| 20210712T10.nm4 |
| 20210712T11.nm4 |
| 20210712T12.nm4 |
| 20210712T13.nm4 |
| 20210712T14.nm4 |
| 20210712T15.nm4 |
| 20210712T16.nm4 |
| 20210712T17.nm4 |
| 20210712T18.nm4 |
| 20210712T19.nm4 |
| 20210712T20.nm4 |
| 20210712T21.nm4 |
| 20210712T22.nm4 |
| 20210712T23.nm4 |
+---20210713
| 20210713T00.nm4 |
Update the values in the settings.json file:
{
"Ais": {
"host": "153.44.253.27",
"port": "5631",
"retryAttempts": 100,
"retryPeriodicity": "00:00:00:00.500"
},
"Storage": {
"connectionString": "<YOUR AZURE STORAGE CONNECTION STRING>",
"containerName": "nmea-ais",
"writeBatchSize": 500
}
}From the command line: dotnet Ais.Net.Receiver.Host.Console.exe
As the AIS.NET stack is written in .NET 6.0 and .NET Standard you can publish the Ais.Net.Receiver.Host.Console application with a target runtime of Portable. This will allow you to run the receiver on a Raspberry Pi if you want to capture your own AIS data.
For reliability you can run Ais.Net.Receiver.Host.Console.dll as daemon.
The combination of Windows Terminal, .NET and PowerShell make a Raspberry Pi a very productive environment for .NET Devs.
Install Windows Terminal. You can download Windows Terminal from the Microsoft Store or from the GitHub releases page.
Open Windows Terminal and use ssh pi@<Raspberry PI IP Address> to connect to your Pi.
Use the following commands to install .NET 6.0 on your Pi.
curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin --channel Currentecho 'export DOTNET_ROOT=$HOME/.dotnet' >> ~/.bashrcecho 'export PATH=$PATH:$HOME/.dotnet' >> ~/.bashrcsource ~/.bashrcdotnet --version
Use the following commands to install PowerShell on your Pi.
- Download the latest package
wget https://github.com/PowerShell/PowerShell/releases/download/v7.2.7/powershell-7.2.7-linux-arm32.tar.gz - Create a directory for it to be unpacked into
mkdir ~/powershell - Unpack
tar -xvf ./powershell-7.2.7-linux-arm32.tar.gz -C ~/powershell - Give it executable rights
sudo chmod +x /opt/microsoft/powershell/7/pwsh - Create a symbolic link
sudo ln -s /opt/microsoft/powershell/7/pwsh /usr/bin/pwsh
use the command pwsh to enter the PowerShell session.
- From the solution root, open a command prompt and type
dotnet publish -c Release .\Solutions\Ais.Net.Receiver.sln - Add your Azure Blob Storage Account connection string to
settings.json - Transfer (I use Beyond Compare as it has native SSH support) the contents of
.\Solutions\Ais.Net.Receiver.Host.Console\bin\Release\net5.0\publishto a folder calledaisrin thehome/pidirectory on your Raspberry Pi (assuming you still have the default set up.) - Copy
Solutions\Ais.Net.Receiver.Host.Console.RaspberryPi\aisr.serviceto/lib/systemd/system/aisr.service - run
sudo chmod 644 /lib/systemd/system/aisr.service - run
sudo systemctl enable aisr.service - run
sudo reboot
You can use journalctl -u "aisr" to view the console output of Ais.Net.Receiver.Host.Console.dll
You can use sudo systemctl restart aisr to restart the service.
If you need to look at / edit the deployed aisr.service use sudo nano /lib/systemd/system/aisr.service make your edits then use Ctrl+O and Ctrl+X to save the file and exit.
Use Azure Storage Explorer to browse to where files are captured.
Configuration is read from settings.json and can also be overridden for local development by using a settings.local.json file.
{
"Ais": {
"host": "153.44.253.27",
"port": "5631",
"loggerVerbosity": "Minimal",
"statisticsPeriodicity": "00:01:00",
"retryAttempts": 100,
"retryPeriodicity": "00:00:00:00.500"
},
"Storage": {
"enableCapture": true,
"connectionString": "DefaultEndpointsProtocol=https;AccountName=<ACCOUNT_NAME>;AccountKey=<ACCOUNT_KEY>",
"containerName": "nmea-ais-dev",
"writeBatchSize": 500
}
}These settings control the ReceiverHost and its behaviour.
host: IP Address or FQDN of the AIS Sourceport: Port number for the AIS SourceloggerVerbosity: Controls the output to the console.Quiet= Essential only,Minimal= Statistics only. Sample rate of statistics controlled bystatisticsPeriodicity,Normal= Vessel Names and Positions,Detailed= NMEA Sentences,Diagnostic= Messages and Errors
statisticsPeriodicity: TimeSpan defining the sample rate of statistics to displayretryAttempts: Number of retry attempts when a connection error occursretryPeriodicity: How long to wait before a retry attempt.
These settings control the capturing NMEA sentences to Azure Blob Storage.
enableCapture: Whether you want to capture the NMEA sentences and write them to Azure Blob StorageconnectionString: Azure Storage Account Connection StringcontainerName: Name of the container to capture the NMEA sentences. You can use this to separate a local dev storage container from your production storage container, within the same storage account.writeBatchSize: How many NMEA sentences to batch before writing to Azure Blob Storage.
AIS.Net.Receiver is also available under the Apache 2.0 open source license.
The Data ingested by the AIS.Net.Receiver is licensed under the Norwegian license for public data (NLOD).
This project is sponsored by endjin, a UK based Technology Consultancy which specializes in Data & Analytics, AI & Cloud Native App Dev, and is a .NET Foundation Corporate Sponsor.
We help small teams achieve big things.
We produce two free weekly newsletters:
- Azure Weekly for all things about the Microsoft Azure Platform
- Power BI Weekly for all things Power BI, Microsoft Fabric, and Azure Synapse Analytics
Keep up with everything that's going on at endjin via our blog, follow us on Twitter, YouTube or LinkedIn.
We have become the maintainers of a number of popular .NET Open Source Projects:
And we have over 50 Open Source projects of our own, spread across the following GitHub Orgs:
And the DevOps tooling we have created for managing all these projects is available on the PowerShell Gallery.
For more information about our products and services, or for commercial support of this project, please contact us.
This project has adopted a code of conduct adapted from the Contributor Covenant to clarify expected behaviour in our community. This code of conduct has been adopted by many other projects. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.