Howto:Processing d-tpp using Python: Difference between revisions

if processing actual PDFs to "retrieve" such navigational data procedurally is ever supposed to "fly", I think it would have to be done using OpenCV runnning in a background thread (actually a bunch of threads in a separate process), i.e. using machine learning - basically, feeding it a bunch of manually-annotated PDFs, segmenting each PDF into sub-areas (horizontal/vertical profile, frequencies, identifier etc) and running neural networks.

Basically, such a thing would need to be very modular to be feasible - i.e. parallel processing of the rasterized image on the GPU, to split the chart into known components and retrieve the identifiers, frequencies, bearings etc that way (i.e. would require an OCR stage, too).

It is kind of an interesting problem and it would address a bunch of legal issues, too - just like downloading such data from the web works for a reason, but it would definitely be a rather complex piece of software I believe, and we would want to get people involved familiar with machine learning and computer vision (OpenCV) - it is kinda a superset of doing OCR on approach charts, i.e. not just looking for a character set, but actual document structure and "iconography" for airports, navaids, route markers and so on.

We will be downloading two different AIRAC cycles, i.e. at the time of writing '''1712''' & '''1713''':

* EuroControl [http://www.eurocontrol.int/articles/eurocontrol-regional-charts-erc]

* VATSIM Charts [https://www.vatsim.net/charts/] [https://www.vatsim.net/charts/]

* IVAO Charts [https://xn.ivao.aero/pilot/xn/charts]

== Chart Classification ==

* '''STARs''' - Standard Terminal Arrivals  

* '''IAPs''' - Instrument Approach Procedures

* '''DPs''' - Departure Procedures

{{Note|This will download roughly 4gb of data in ~17k files, for each AIRAC cycle!}}  

* this should support caching

* and interrupting/resuming scraping  

* Alternatively, use a media pipeline <ref>http://sergeis.com/web-scraping/downloading-files-scrapy-mediapipeline/</ref>

{{Note|By default, all PDF files will be  387 x 594 pts (use pdfinfo to see for yourself)}}

pip3 install pdf2image <ref>https://github.com/Belval/pdf2image</ref>

[[File:Random-charts.png|thumb|screenshot showing simplified charts, based on creating thumbnails that are added to a new image.|right ]]

We can easily simplify our charts by creating thumbnails for each chart and merging all files into a larger image/texture:

Since we only have very little data, we need to come up with artifical data fo training purposes - we can do so by randomizing our existing image set to create all sorts of "charts". e.g. by transforming/re-scaling our images or changing their aspect ratio:

We don't just need to do character recognition, but also deal with aviation specific symbology/iconography.

Once again, we can refer to PDF files for the specific symbols <ref>https://www.icao.int/safety/ais-aimsg/AISAIM%20Meeting%20MetaData/AIS-AIMSG%204/SN%208%20Att%20B.pdf</ref>

pip install --user  

== Python resources ==

== See also ==

 

== Related ==

[[Category:Python Software]]