A NavDB web service: Difference between revisions

From FlightGear wiki
Jump to navigation Jump to search
No edit summary
(Redo quotes)
Line 3: Line 3:


== Background ==
== Background ==
{{FGCquote
A user once asked in the forums whether FG wouldsupport navaids not only from the nav.dat.gz file, but from files in custom sceneries.
  |Will FG support navaids not only from nav.dat.gz file, but from some files in custom sceneries?<br/>
For example, in ULLI airport there is a problem with glideslope and NDB at the O0outer markers and there are missing NDBs at middle markers. A user made a correct version of nav.dat.gz, but it worked only on his computer. He could distribute his scenery with the fixed nav.dat.gz, but if another player had made his own modifications in his own nav.dat, they would conflict. It would be possibls to write instructions how to find bad data and replace it with good data, but not every player wants to do such things, especially, if he has played MSFS earlier. That the main problem.<ref>{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=174016#p174016
For example, in ULLI airport there is problem with glideslope and NDB at Outer markers and there are missing NDB at Middle markers. I made right version of nav.dat.gz, but it works only on my computer. I can distribute my scenery with fixed nav.dat.gz, but if player have own modifications in own nav.dat, they will conflict. I can write instruction how to find bad data and replace it to right, but not every player wants to do some actions, especially, if he played MSFS earlier. That why I'm asking it.
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=174016#p174016
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |author=<nowiki>Soitanen</nowiki>
     |author=<nowiki>Soitanen</nowiki>
     |date=<nowiki>Sun Jan 06</nowiki>
     |date=<nowiki>Sun Jan 06</nowiki>
   }}
   }}</ref>
}}


== Objective ==
== Objective ==
{{FGCquote
At least for the US, there's a ton of data freely available that can be extracted from such charts procedurally using a bit of scripting and OCR or regex. The corresponding data could be put into a database and updated regularly, according to the AIRAC cycle. But something like that should really be a central web service - in FG, we now have the means to easily download data via HTTP, and make web service requests. In other words, we could populate some kind of central navdb with our existing data, update the whole thing with the kind of data provided by the scripted discussed at [http://forum.flightgear.org/viewtopic.php?p{{=}}206529#p206529 Subject: A project to create a source of free geo-referenced instrume].
  |At least for the US, there's a ton of data freely available that can be extracted from such charts procedurally using a bit of scripting and OCR or regex. The corresponding data could be put into a database and updated regularly, according to the AIRAC cycle. But something like that should really be a central web service - in FG, we now have the means to easily download data via HTTP, and make web service requests. In other words, we could populate some kind of central navdb with our existing data, update the whole thing with the kind of data provided by the scripted discussed at [http://forum.flightgear.org/viewtopic.php?p{{=}}206529#p206529 Subject: A project to create a source of free geo-referenced instrume].<br/>
 
<br/>
At that point, we would have a web service that could run a cron job to regularly update the whole thing - maybe in combination with some kind of wiki-like front-end to allow contributors to update/edit and maintain entries for different countries and navaids, as well as procedures.<br/>
At that point, we would have a web service that could run a cron job to regularly update the whole thing - maybe in combination with some kind of wiki-like front-end to allow contributors to update/edit and maintain entries for different countries and navaids, as well as procedures.<br/>
<br/>
 
The navdb would then be md5-hashed and fetched on demand, so that the navcache can be updated/rebuilt accordingly.
The navdb would then be md5-hashed and fetched on demand, so that the navcache can be updated/rebuilt accordingly. <ref> {{cite web |url=http://forum.flightgear.org/viewtopic.php?p=213149#p213149
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=213149#p213149
     |title=<nowiki>Re: 777 EFB: initial feedback</nowiki>
     |title=<nowiki>Re: 777 EFB: initial feedback</nowiki>
     |author=<nowiki>Hooray</nowiki>
     |author=<nowiki>Hooray</nowiki>
     |date=<nowiki>Sun Jun 22</nowiki>
     |date=<nowiki>Sun Jun 22</nowiki>
   }}
   }} </ref>
}}
 
== EFB ==
For a complex EFB, it would be possible to create 100% of the charts procedurally, including charts for TPs (terminal procedures like SIDs/DPs, STARs and IAPs) - the main issue is lack of data. And even if we get our hands on a relatively recent AIRAC cycle, the FG side of things (navdb) will not match that data. And in  FG, we don't currently have the corresponding TP data. Converting/rendering existing charts (even PDFs from airnav.com)  should be straightforward, but the FG world would use a different set of navaids and FIXES. So, we'd be better off looking at how X-Plane handles this challenge, given that they also used to use DAFIF(T) originally.
 
In general, developers would probably favor procedural creation if we could get the data - and short of that, I'd use the approach discussed a while ago: [http://forum.flightgear.org/viewtopic.php?p{{=}}206529#p206529 Subject: A project to create a source of free geo-referenced instrume]<br/>


{{FGCquote
Such charts would be centrally created, along with VRT (XML) files for geo-referencing them, so that the corresponding charts could be requested at run-time and downloaded on demand (or cached).
  |like you said, we can create 100% of the charts procedurally, including charts for TPs (terminal procedures like SIDs/DPs, STARs and IAPs) - the main issue is lack of data. And even if we get our hands on a relatively recent AIRAC cycle, the FG side of things (navdb) will not match that data. And in  FG, we don't currently have the corresponding TP data. Converting/rendering existing charts (even PDFs from airnav.com)  should be straightforward, but the FG world would use a different set of navaids and FIXES. So, we'd be better off looking at how X-Plane handles this challenge, given that they also used to use DAFIF(T) originally.<br/>
We could probably use a simple form of OCR to extract navaid info from such charts to obtain relevant navaids for the given sector - it's just our navdb design that is kind of inflexible at the moment, but we could certain enrich/augment existing data using such updated data for navaids and fixes (or procedures).<ref> {{cite web |url=http://forum.flightgear.org/viewtopic.php?p=213143#p213143
<br/>
In general, I'd probably favor procedural creation if we can get the data - and short of that, I'd use the approach discussed a while ago: [http://forum.flightgear.org/viewtopic.php?p{{=}}206529#p206529 Subject: A project to create a source of free geo-referenced instrume]<br/>
<br/>
<br/>
Such charts would be centrally created, along with VRT (XML) files for geo-referencing them, so that the corresponding charts could be requested at run-time and downloaded on demand (or cached).<br/>
We could probably use a simple form of OCR to extract navaid info from such charts to obtain relevant navaids for the given sector - it's just our navdb design that is kind of inflexible at the moment, but we could certain enrich/augment existing data using such updated data for navaids and fixes (or procedures)
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=213143#p213143
     |title=<nowiki>Re: 777 EFB: initial feedback</nowiki>
     |title=<nowiki>Re: 777 EFB: initial feedback</nowiki>
     |author=<nowiki>Hooray</nowiki>
     |author=<nowiki>Hooray</nowiki>
     |date=<nowiki>Sun Jun 22</nowiki>
     |date=<nowiki>Sun Jun 22</nowiki>
   }}
   }}  
}}
</ref>


== Status ==
== Status ==
{{FGCquote
[http://navdata.fgx.ch/ http://navdata.fgx.ch/]{{dead link}}
  | [http://navdata.fgx.ch/ http://navdata.fgx.ch/]{{dead link}}<br/>
 
{{cquote|The NavData project takes the latest xplane data, imports it to a postgis database spatially ready for querying on demand. It is intended as a "permanent" feed moving foward and making the latest xplane data available and updated for all, whether for caching local or online live. The data returned is formatted in a pratical way with a lot of the calculations already done, such as the middle of an airport, runway center point, orientation and displaced threshold etc. Also measures are returned in different units such as evelation both in metres and feet to make life easier for both calculation and presentation for end user application.}}
The NavData project takes the latest xplane data, imports it to a postgis database spatially ready for querying on demand. It is intended as a "permanent" feed moving foward and making the latest xplane data available and updated for all, whether for caching local or online live. The data returned is formatted in a pratical way with a lot of the calculations already done, such as the middle of an airport, runway center point, orientation and displaced threshold etc. Also measures are returned in different units such as evelation both in metres and feet to make life easier for both calculation and presentation for end user application. <ref>{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=207549#p207549
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=207549#p207549
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |author=<nowiki>Hooray</nowiki>
     |author=<nowiki>Hooray</nowiki>
     |date=<nowiki>Tue Apr 29</nowiki>
     |date=<nowiki>Tue Apr 29</nowiki>
   }}
   }} </ref>
}}
 
== Developer's Plans ==
We quote ac001: 'I'm the developer behind navdata.fgx.c and thanks to Hooray for alerting me as i dont follow the forums much...
 
So What I would want is to publich the xplane data.
 
But also have a way to "submit corrections", and maybe have a voting system of "yes this is correct".....
 
Problem is to create and automated way to send this upstream to Robin the xplane maintainer.. but am well into this stuff


{{FGCquote
Problem with navdata at the moment is I need some users to check the api and make it "usable" moving forward...' <ref>{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=207675#p207675
  |I'm the developer behind navdata.fgx.ch<br/>
and thanks to Hooray for alerting me as i dont follow the forums much...<br/>
<br/>
So What I would want is to publich the xplane data..<br/>
<br/>
But also have a way to "submit corrections", and maybe have a voting system of "yes this is correct".....<br/>
<br/>
Problem is to create and automated way to send this upstream to Robin the xplane maintainer...<br/>
<br/>
but am well into this stuff ;-)<br/>
<br/>
Problem with navdata at the moment is I need some users to check the api and make it "usable" moving forward...
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=207675#p207675
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |author=<nowiki>ac001</nowiki>
     |author=<nowiki>ac001</nowiki>
     |date=<nowiki>Wed Apr 30</nowiki>
     |date=<nowiki>Wed Apr 30</nowiki>
   }}
   }} </ref>
}}


{{FGCquote
== Coordinating with XPlane ==
  |coordinating this with Robin Peel would be awesome, because of the sizable XP community, i.e. we could truly benefit here by leveraging our two communities to help update such data.<br/>
Coordinating this with Robin Peel would be awesome, because of the sizable XP community, i.e. we could truly benefit here by leveraging our two communities to help update such data.
At some point, someone even came up with a script to extract such stuff procedurally from airnav/TPP charts. This is something that me worthwhile to explore again.<br/>
At some point, someone even came up with a script to extract such stuff procedurally from airnav/TPP charts. This is something that me worthwhile to explore again.
API-wise I'd just suggest to provide a mandatory version attribute, so that you can safely upgrade things later on, without features breaking.<br/>
API-wise, it would be a good idea to provide a mandatory version attribute, so that you can safely upgrade things later on, without features breaking. <ref>  
<br/>
{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=207695#p207695
Also see: [http://sourceforge.net/p/flightgear/mailman/flightgear-devel/thread/536E6B56.2030209%40btinternet.com/#msg32324567 http://sourceforge.net/p/flightgear/mai ... sg32324567]
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=207695#p207695
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |author=<nowiki>Hooray</nowiki>
     |author=<nowiki>Hooray</nowiki>
     |date=<nowiki>Wed Apr 30</nowiki>
     |date=<nowiki>Wed Apr 30</nowiki>
   }}
   }} </ref>
}}
 
Also see: [http://sourceforge.net/p/flightgear/mailman/flightgear-devel/thread/536E6B56.2030209%40btinternet.com/#msg32324567 http://sourceforge.net/p/flightgear/mai ... sg32324567]


{{FGCquote
Robin is already working on improving the submission side (of apt.dat at least). See [http://developer.x-plane.com/2013/12/airport-authoring-sharing-airports/ http://developer.x-plane.com/2013/12/ai ... -airports/] <ref> {{cite web |url=http://forum.flightgear.org/viewtopic.php?p=207698#p207698
  |Robin's already working on improving the submission side (of apt.dat at least). See [http://developer.x-plane.com/2013/12/airport-authoring-sharing-airports/ http://developer.x-plane.com/2013/12/ai ... -airports/]  
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=207698#p207698
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |author=<nowiki>Gijs</nowiki>
     |author=<nowiki>Gijs</nowiki>
     |date=<nowiki>Wed Apr 30</nowiki>
     |date=<nowiki>Wed Apr 30</nowiki>
   }}
   }}  
}}
</ref>


== A NavDB wiki ? ==
== A NavDB wiki ? ==


{{FGCquote
If you know:
  |If you know:<br/>
* someone who is willing to develop, contruct and maitain the full airport diagrams database + its implementation;
* someone who is willing to develop, contruct and maitain the full airport diagrams database + its implementation ;<br/>
* a solution better than the formats used in XPlane;
* a solution better than the formats used in XPlane ;<br/>
* someone who has the time to rebuild the whole toolchain to analyse this new format;
* someone who has the time to rebuild the whole toolchain to analyse this new format ;<br/>
* build a nice and crossplatform tool we can use to build airports;
* build a nice and crossplatform tool we can use to build airports ;<br/>
* with a GPL license.
* with a GPL license.<br/>
 
<br/>
Then, we'd willing to hear about him - or her. <ref>{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=152714#p152714
Then, I'm willing to hear about him - or her.<br/>
<br/>
I don't really understand what you mean by "better solution" and "more freedom" regarding the airport data format.
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=152714#p152714
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |author=<nowiki>f-ojac</nowiki>
     |author=<nowiki>f-ojac</nowiki>
     |date=<nowiki>Thu Mar 08</nowiki>
     |date=<nowiki>Thu Mar 08</nowiki>
   }}
   }}
}}
</ref>
 
== Advantages of staying with XPlane ==
There are so many advantages when reusing the X-Plane data and their format, they have a HUGE community of contributors - which FG simply doesn't have.
 
On the other hand, some web-based frontend to allow people to easily contribute and maintain airport/navdb data, would be great to have.
Something pretty much like a "GIS Wiki", possibly based on PSQL or even git - so that people could go to a website, open an airport/navaid and file merge requests, review entries and help maintain all the data there would be useful.
 
The new [http://scenemodels.flightgear.org/ scenemodels] submission process is actually a very good example here.


{{FGCquote
  |There are so many advantages when reusing the X-Plane data and their format, they have a HUGE community of contributors - which FG simply doesn't have.<br/>
<br/>
On the other hand, some web-based frontend to allow people to easily contribute and maintain airport/navdb data, would be great to have in my opinion: <br/>
Pretty much like a "GIS Wiki", possibly based on PSQL or even git - so that people could go to a website, open an airport/navaid and file merge requests, review entries and help maintain all the data there. <br/>
<br/>
The new [http://scenemodels.flightgear.org/ scenemodels] submission process is actually a very good example here.<br/>
<br/>
Something like this could be very useful, not just for airports (runways, taxiways, aprons etc), but also for navaids and terminal procedures (SIDs/STARs and IAPs).<br/>
Something like this could be very useful, not just for airports (runways, taxiways, aprons etc), but also for navaids and terminal procedures (SIDs/STARs and IAPs).<br/>
<br/>
 
And I guess we could even get the X-Plane community to contribute to something like this.<br/>
And I guess we could even get the X-Plane community to contribute to something like this.
The whole work flow could be moved to a website, so that it would be easier contribute.<br/>
The whole work flow could be moved to a website, so that it would be easier contribute.
<br/>
 
Using git as the backend (i.e. plain text files) would have the advantage, that we'd also get a revision history automatically.
Using git as the backend (i.e. plain text files) would have the advantage, that we'd also get a revision history automatically.<ref>{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=152722#p152722
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=152722#p152722
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
     |author=<nowiki>Hooray</nowiki>
     |author=<nowiki>Hooray</nowiki>
     |date=<nowiki>Thu Mar 08</nowiki>
     |date=<nowiki>Thu Mar 08</nowiki>
   }}
   }}
}}
</ref>
 
We need to learn the Open Street Map lesson here. <!-- expand -->


{{FGCquote
<!-- == ATC Clients ==
  |I totally agree with that. The OpenStreetMap lesson has to be learned here
  |{{cite web |url=http://forum.flightgear.org/viewtopic.php?p=153309#p153309
    |title=<nowiki>Re: Why does Flightgear use X-Plane airport data?</nowiki>
    |author=<nowiki>robitabu</nowiki>
    |date=<nowiki>Fri Mar 16</nowiki>
  }}
}}


== ATC Clients ==
{{FGCquote
|1= All navaids, airports and even the runways can be displayed. Biggest flaw is the absence of the taxiways. I will check if I can do something for that too...
|2= {{cite web
  | url    = http://forum.flightgear.org/viewtopic.php?p=264689#p264689
  | title  = <nowiki>Improving stability: No-scenery-emergency mode</nowiki>
  | author = <nowiki>wagnerw</nowiki>
  | date  = Nov 16th, 2015
  }}
}}


{{FGCquote
{{FGCquote
Line 165: Line 127:
   }}
   }}
}}
}}
REALLY NECESSARY? -->
== References ==


[[Category:RFC]]
[[Category:RFC]]

Revision as of 01:32, 24 March 2016

This article is a stub. You can help the wiki by expanding it.
WIP.png Work in progress
This article or section will be worked on in the upcoming hours or days.
See history for the latest developments.

Background

A user once asked in the forums whether FG wouldsupport navaids not only from the nav.dat.gz file, but from files in custom sceneries. For example, in ULLI airport there is a problem with glideslope and NDB at the O0outer markers and there are missing NDBs at middle markers. A user made a correct version of nav.dat.gz, but it worked only on his computer. He could distribute his scenery with the fixed nav.dat.gz, but if another player had made his own modifications in his own nav.dat, they would conflict. It would be possibls to write instructions how to find bad data and replace it with good data, but not every player wants to do such things, especially, if he has played MSFS earlier. That the main problem.[1]

Objective

At least for the US, there's a ton of data freely available that can be extracted from such charts procedurally using a bit of scripting and OCR or regex. The corresponding data could be put into a database and updated regularly, according to the AIRAC cycle. But something like that should really be a central web service - in FG, we now have the means to easily download data via HTTP, and make web service requests. In other words, we could populate some kind of central navdb with our existing data, update the whole thing with the kind of data provided by the scripted discussed at Subject: A project to create a source of free geo-referenced instrume.

At that point, we would have a web service that could run a cron job to regularly update the whole thing - maybe in combination with some kind of wiki-like front-end to allow contributors to update/edit and maintain entries for different countries and navaids, as well as procedures.

The navdb would then be md5-hashed and fetched on demand, so that the navcache can be updated/rebuilt accordingly. [2]

EFB

For a complex EFB, it would be possible to create 100% of the charts procedurally, including charts for TPs (terminal procedures like SIDs/DPs, STARs and IAPs) - the main issue is lack of data. And even if we get our hands on a relatively recent AIRAC cycle, the FG side of things (navdb) will not match that data. And in FG, we don't currently have the corresponding TP data. Converting/rendering existing charts (even PDFs from airnav.com) should be straightforward, but the FG world would use a different set of navaids and FIXES. So, we'd be better off looking at how X-Plane handles this challenge, given that they also used to use DAFIF(T) originally.

In general, developers would probably favor procedural creation if we could get the data - and short of that, I'd use the approach discussed a while ago: Subject: A project to create a source of free geo-referenced instrume

Such charts would be centrally created, along with VRT (XML) files for geo-referencing them, so that the corresponding charts could be requested at run-time and downloaded on demand (or cached). We could probably use a simple form of OCR to extract navaid info from such charts to obtain relevant navaids for the given sector - it's just our navdb design that is kind of inflexible at the moment, but we could certain enrich/augment existing data using such updated data for navaids and fixes (or procedures).[3]

Status

http://navdata.fgx.ch/[dead link]

The NavData project takes the latest xplane data, imports it to a postgis database spatially ready for querying on demand. It is intended as a "permanent" feed moving foward and making the latest xplane data available and updated for all, whether for caching local or online live. The data returned is formatted in a pratical way with a lot of the calculations already done, such as the middle of an airport, runway center point, orientation and displaced threshold etc. Also measures are returned in different units such as evelation both in metres and feet to make life easier for both calculation and presentation for end user application. [4]

Developer's Plans

We quote ac001: 'I'm the developer behind navdata.fgx.c and thanks to Hooray for alerting me as i dont follow the forums much...

So What I would want is to publich the xplane data.

But also have a way to "submit corrections", and maybe have a voting system of "yes this is correct".....

Problem is to create and automated way to send this upstream to Robin the xplane maintainer.. but am well into this stuff

Problem with navdata at the moment is I need some users to check the api and make it "usable" moving forward...' [5]

Coordinating with XPlane

Coordinating this with Robin Peel would be awesome, because of the sizable XP community, i.e. we could truly benefit here by leveraging our two communities to help update such data. At some point, someone even came up with a script to extract such stuff procedurally from airnav/TPP charts. This is something that me worthwhile to explore again. API-wise, it would be a good idea to provide a mandatory version attribute, so that you can safely upgrade things later on, without features breaking. [6]

Also see: http://sourceforge.net/p/flightgear/mai ... sg32324567

Robin is already working on improving the submission side (of apt.dat at least). See http://developer.x-plane.com/2013/12/ai ... -airports/ [7]

A NavDB wiki ?

If you know:

  • someone who is willing to develop, contruct and maitain the full airport diagrams database + its implementation;
  • a solution better than the formats used in XPlane;
  • someone who has the time to rebuild the whole toolchain to analyse this new format;
  • build a nice and crossplatform tool we can use to build airports;
  • with a GPL license.

Then, we'd willing to hear about him - or her. [8]

Advantages of staying with XPlane

There are so many advantages when reusing the X-Plane data and their format, they have a HUGE community of contributors - which FG simply doesn't have.

On the other hand, some web-based frontend to allow people to easily contribute and maintain airport/navdb data, would be great to have. Something pretty much like a "GIS Wiki", possibly based on PSQL or even git - so that people could go to a website, open an airport/navaid and file merge requests, review entries and help maintain all the data there would be useful.

The new scenemodels submission process is actually a very good example here.

Something like this could be very useful, not just for airports (runways, taxiways, aprons etc), but also for navaids and terminal procedures (SIDs/STARs and IAPs).

And I guess we could even get the X-Plane community to contribute to something like this. The whole work flow could be moved to a website, so that it would be easier contribute.

Using git as the backend (i.e. plain text files) would have the advantage, that we'd also get a revision history automatically.[9]

We need to learn the Open Street Map lesson here.


References