Responding to Richard

[IAHM-COL]

Sorry for the slow reply; this has taken me days to write

Thanks for answering.
I was starting to worry you've left the room, uninterested.


Now for your reply, I am starting to worry we are not discussing in the same table, nor on the same language, at all. But I'll give it a try.
Right now I feel we are expanding more and more beyond the point, and spreading ourselves way too thin.

[IAHM-COL]

What I think of as the scenery database is the sources that are used to build the run time scenery database which I think of (and might refer to) as the visual database.

Do you refer to https://scenery.flightgear.org/?
Because that description is totally non-apt for that database.

This is the scenery website does not contain sources. That'll be quite ideal.
You are starting to think to much in terms of sources -> compile -> binary asset.

But this is not how this scenery construction for FG actually is working.

As an example, would you find the sources in .blend and .xcf or .svg (or whatever source applies) for the graphic assets .ac and .png in the website database? I am certain you would not. What you would find is the actual .ac .xml and .png files that will actually just be copied into the "visual database" that you call.
In reality their contents (both the website and the "visual" database) is identical, for their formats differ. Whereas one can be interpreted by FG as is (the visual) the other needs to be re-parsed, and files transferred to a directory structure that end up making sense to FG.

So, trying to sell the Website database as a "sources database" is a bit of an over-sellout.

[IAHM-COL]

1. Is the numeric ID in the STG guaranteed never to change?

example
https://github.com/FGMEMBERS-TERRAGIT/w ... 892922.stg
<snip>

Code: Select all

#General Parking
OBJECT_STATIC park1.xml -3.375314   55.946157   29.60   0 0
OBJECT_STATIC park2.xml -3.36986684 55.94360593 31.8241 0 0
OBJECT_STATIC park3.xml -3.36517635 55.94562957 31.0069 0 0
OBJECT_STATIC park4.xml -3.36453017 55.94622915 29.8322 0 0
OBJECT_STATIC park5.xml -3.35274418 55.94548226 32.1660 -3 0


No numeric Index ID present. The STG database is parsed by FG one line at a time, and it draws the specified object in that one line in the lat, lon, elevation, heading, pitch coordinates presented.

[IAHM-COL]

2. What is your opinion of the bulk upload facility; is it something that could be used with modifications or is it not suitable. A brief glance through seems it's only for shared objects, but there are quite a lot of them in 2892922.stg; and maybe some of the static objects could be shared? (I have no idea what the implications of this would be)

In my opinion, the FlightGear scenery website should be re-purposed.
I don't think it is worth a usage for Gateway (entry) of content. It could be used for catalogue of assets in the World Scenery. So instead of being an web-input of content could be a searchable index of content.
This way other resources that use it, like the objects map, and the KL666 map of developed airports could systematically explore the content on that database.

Also, this would imply that such database must be filled automatically, by just reading the content of the Scenery directory (in terrasync) as opposed to be used to create/alter/transform/edit such content. [Preventing unintended deletions of objects, per example]

See Torsten Dreyer argument of why terrasync cannot ship OSM2city buildings

https://sourceforge.net/p/flightgear/ma ... /35355688/

I can only add the terrain. Models and their Objects are generated from the
scenemodels DB.
As I can see, there is one shared model: maledivianhouse.ac and many
osm2city generated static objects.
I don't think we have found a generic way how to deal with those. I have
the gut feeling that the scenemodels DB is not the best place to store
auto-generated objects.
However the entire Objects-Tree is generated from the database and whatever
I put there manually gets overwritten by the next database update.

What I could do quickly is adding your terrain. For the generated osm2city
models, we need to think about a general solution how to handle those.

Torsten

As you see above, this method of using the database is actually creating us problems to implement solutions. I don't see the motivation to defend such thing.

So. In general, my posture is, the Website database IS NOT suitable to populate the terrasync scenery. That is convenient to be stopped. Instead, populate the Website database with the content existing -- which is exactly the opposite direction for information flow.

Second part: Shared or not shared

Shared and non-shared (static) objects should be non-interchangeable in fact.

A shared object should be reserved for those objects with large number of repetitions in the World Scenery, and that are used throughout the World (scenery). Tipically, they are used in several (more than a few) tiles. Example: A lampost, or a generic Hangar or an static aircraft. You can reload the same object in thousands of aiports at the same time. D-LASER P3000 contains a great example of a large and beautiful collection of Shared objects, which are used quite properly as SHARED.

https://github.com/FGMEMBERS-TERRAGIT/t ... Models/lib
https://github.com/FGMEMBERS-SCENERY/Project3000

An static object is reserved for all those objects that are unique to a very discrete number of iterations in the world, and tipically those are single locations. Think, per example, the Liberty Lady Statue in NY. You wont find thousand of these models around the Globe.
But also terminal buildings are tipically quite unique. And thus static.
Or if I build a conglomerate of buildings, modelling the General Aviation apron of Edinburgh, well... then such collection becomes quite unique, and thus it is not shared.

Computationally, the way that memory is allocated for shared and static models in flightGear differs. In particularly, once loaded, a shared object remains in the cache until the program exits. And off course, this model is reused as specified in the STG. An static model can be deallocated, as soon as by LOD-bare is no longer needed. It makes sense to respect the nature (SHARED or STATIC) of each object. (instead of making shared objects for a "database" purpose or for uploading to a gateway purpose)

[IAHM-COL]

From your list
>2. Scenery is the configuration files for airports, (xml files) are not specified in the STG database.
>3. Artificial Intelligence traffic files: are not specified in the STG database

It seems to me that these really should be stored in a database; but I'm guessing that (2) above is probably a result of other items that could be in a database or built as a result of some process, and (3) above is probably quite complex and might not be that easy to store.

You are correct on this one.
Particularly I think that a direction that FG core developers should be actually working really hard on right now is on re-implementing the apt.dat parser

They seem to be busy instead of a launcher. Which saddens me. A launcher is not core.

Let me give you a very overhead of this problem, since it has many interesting situations, but again we are driving ourselves too thin, right now.

The X-Plane specification of apt.dat have grown more complex over time. Most of the airport xml files content is actually developed and present inside the apt.dat specification. This includes

* tower location
* groundnet (required for AI)
* atc network (required for AI)
* taxiways required for AI)
* runway threshold

In reality, the usage of XML files to configure these things for airports in FG should be totally deprecated right now, and instead, the apt.dat parser in FG should be optimized so it reads all of this information directly from the corresponding apt.dat source.
that solves the issue of having this information on a "database"

---since the apt.dat specification for aiports IS a database!

In reality, the correct database to configure airports.

Secondly, FG should be now accepting Florent's pull request for FG to be able to use multiple apt.dat sources redirecteable using --apt.dat=/path/to/additional/apt.dat files.
This would be a great step forward in this Scenery development thing. And Florents pull request is actually very bright and clever.

So back again,
The XML contents of Airports/ directory does not belong to "the Website" database. Instead, its usage should be deprecated in favor of better parsing the apt.dat

And finally, the apt.dat should be, if they are going to be at it, uncompressed, as in text plain format, and maybe used as I once suggested a few years ago, splitting it into smaller apt.dat files.

[IAHM-COL]

The terrain mesh can remain as binary files that are computed. I believe the original plan (for scenery.flightgear) was to somehow include the terrain mesh so that the XPlane database changes could be picked up on demand; but I'm not sure I'm right about this. The fact that the terrain mesh hasn't changed in the last few years isn't really a good guide as to what might happen in the future as this all depends on what happens with the cutting out of airports. If this had been automated for a few years the git size could have grown a lot. I don't know, and actually nobody knows this until it happens in the future - but we need to consider the future when thinking about how things should be done. So in many ways what's been happening with the mesh and other binary assets is a bad guide to what might be needed in the future; but bear in mind that I'm not against using git where it works well - but at the same point git repositories do get slower over time, both to download and for certain operations (such as blame)

And that's exactly why terraGIT is proposed under the submodules alternative. This way nothing of this effort is loaded onto a single repository. Editing an airport in Russia (on a corresponding tiled repository) has Zero effect on the content, flexibility and history of the git repository of A certain part of Central America, per example.

Every tile contains a much limited number of airports that could change... or tiles that could be edited, per example.

It is actually very modular, addressing this valid concern.

[IAHM-COL]

Merging STG files and lumping objects together at run time is a good thing; I just don't think that merged scenery should be submitted to the source scenery database as it then becomes is impossible (or really hard) to undo a merge; whereas it is easy to perform a merge.

It is a problem of granularity

Example.
There is not much gain from

ID=1058, Edingburgh_cargoArea_Object

to

ID=1058, Edingburgh_cargoArea_buiding1
ID=1059, Edingburgh_cargoArea_buiding2
ID=1060, Edingburgh_cargoArea_buiding3
ID=1061, Edingburgh_cargoArea_buiding4
ID=1062, Edingburgh_cargoArea_buiding5
ID=1063, Edingburgh_cargoArea_buiding6
ID=1064, Edingburgh_cargoArea_buiding7
ID=1065, Edingburgh_cargoArea_buiding8
ID=1066, Edingburgh_cargoArea_buiding9
ID=1067, Edingburgh_cargoArea_buiding10
ID=1068, Edingburgh_cargoArea_buiding11
ID=1069, Edingburgh_cargoArea_buiding12

In reality, that much granularity on the database content becomes a "no longer a gain" zone

But
you will need, texture file per building, a lightmap per building, and a xml configurer per building.

So in the "pain zone" you get lots, for the developer.
Who also needs to now carefully place more objects.

It becomes a madness of degrees of freedom.

The stgmerge now also, is rather rudimentary, and per example only is useful for ac files. So objects more developed with lightmaps and animations? It is not going to happen,

Look. Let's not fight about the database rules. They are not optimized, and there is no defense you can make on their favor, really.

[IAHM-COL]

I thought the bulk upload was to allow more things to be submitted at once.

You've thought wrong.
only shared objects locations you can do that. And only if the shared object already exists in the database.
For a new shared object, you have to do the whole deal on the database. Which in the case of this particular database is Lots of time formatting the object solely to fit the database requirements.

Not for really functional purposes.

[IAHM-COL]

When I say "one problem" I mean only one problem because everything else is working fine.

It's Just that it does not.

Are you saying that updating airports, terrain, elevations, airport configuration files, jetways in FG works just fine?
Are you saying that uploading new shared or new static objects to the database to push them to terrasync works just fine?
Are you saying the WS2.0 scenery for flightgear with the sunk and underground airports everywhere works just fine?

We are not even in the same negotiation table Richard. Let alone speaking the same language.

My language is that of an impending need to change how we are populating the Terrasync World. An impending need to facilitate users to add improvements, and facilitate those improvements to reach the community as a whole

Your language... apparently is: All is fine. Except that submitting improvements is just a little bit difficult.

Well... it is not difficult. In the great majority of situations, as things are right now, is actually undoable.

[IAHM-COL]

My fear is just based on experience that it is better to control the source rather than the generated or built runtimes.

We definitely are not talking about the same beast (a bit frustated)

Lets review our sources.

Source for Terrain

Flightgear uses SRTM data for creating the terrain meshes. Terragear can take any SRTM data (at 1 or 3 arc-degrees resolution)... but in reality it standard uses the most curated of all available under GPL. This sources of terrain are found and distributed by viewfinder panorama, here

http://viewfinderpanoramas.org/Coverage ... s_org3.htm

And Flightgear does not control it. But if you ask me, it would not be better for us to attempt at controlling that. We are better off running downstream of that content.

Source for Landclasses

Flightgear uses either V0 Landclasses, or CORINE _for Europe_ to decode the Scenery landclasses.

And Flightgear does not control these But if you ask me, it would not be better for us to attempt at controlling that. We are better off running downstream of that content.


Source for Airport specifications

Flightgear uses X-PLANE Apt.dat database. No, we dont have any control of these sources... either. And I think the x-plane community is doing a superb job editing and improving this specification, much faster than we achieve to just copy and use the content.

I think Flightgear is much better set up also not trying to control these.