Sorry for the slow reply; this has taken me days to write.
What I think of as the scenery database is the sources that are used to build the run time scenery database which I think of (and might refer to) as the visual database.
So we have two databases; the source data that contributors create is really what we need to take care with for the future; the run time database can (and probably will) be regenerated from the sources.
IAHM-COL wrote:At this point, I am limited to bow to your knowledge in computational science and Engineer, but it seems from my experience within FG, that it was an sloppy approach all from the beginning.
Thanks, but feel free to challenge anything I say as I'm not always right. In this instance separating the source from the target data is a good thing in my opinion because it adds a layer between the authored content and the run time - and in this layer any manipulations that are needed could be performed. Equally it becomes possible to write something to generate a completely different (e.g. OpenFlight) database from the source scenery database.
Looking at the XML these are my questions:
1. Is the numeric ID in the STG guaranteed never to change?
2. What is your opinion of the bulk upload facility; is it something that could be used with modifications or is it not suitable. A brief glance through seems it's only for shared objects, but there are quite a lot of them in 2892922.stg; and maybe some of the static objects could be shared? (I have no idea what the implications of this would be)
From your list
>2. Scenery is the configuration files for airports, (xml files) are not specified in the STG database.
>3. Artificial Intelligence traffic files: are not specified in the STG database
It seems to me that these really should be stored in a database; but I'm guessing that (2) above is probably a result of other items that could be in a database or built as a result of some process, and (3) above is probably quite complex and might not be that easy to store.
The terrain mesh can remain as binary files that are computed. I believe the original plan (for scenery.flightgear) was to somehow include the terrain mesh so that the XPlane database changes could be picked up on demand; but I'm not sure I'm right about this. The fact that the terrain mesh hasn't changed in the last few years isn't really a good guide as to what might happen in the future as this all depends on what happens with the cutting out of airports. If this had been automated for a few years the git size could have grown a lot. I don't know, and actually nobody knows this until it happens in the future - but we need to consider the future when thinking about how things should be done. So in many ways what's been happening with the mesh and other binary assets is a bad guide to what might be needed in the future; but bear in mind that I'm not against using git where it works well - but at the same point git repositories do get slower over time, both to download and for certain operations (such as blame).
Merging STG files and lumping objects together at run time is a good thing; I just don't think that merged scenery should be submitted to the source scenery database as it then becomes is impossible (or really hard) to undo a merge; whereas it is easy to perform a merge.
I thought the bulk upload was to allow more things to be submitted at once.
IAHM-COL wrote:It is only solving one problem - which is the difficulty in getting scenery submitted;
THAT IS THE PROBLEM.
When I say "one problem" I mean only one problem because everything else is working fine. So if too much infrastructure is changed without proper consideration then other things that are working well may no longer work; and maybe not immediately in an obvious way.
IAHM-COL wrote:But your fear, I do not comprehend.
Once FG releases FG Scenery V3 (the terrain) we simply update TerraGIT Terrain branches (or possibly even better, split the content in 2 repositories one for terrain other for Objects).
My fear is just based on experience that it is better to control the source rather than the generated or built runtimes.
For example; I think the current scheme has survived the V1 to V2 transition so it should survive going to V3; whereas I'm genuinely not sure about what's going to happen with git when it comes to a merge with a newly generated V3 database. Obviously I'd hope it went well and git managed to sort out any conflicts or merges; but until it's been done we just don't know what will happen. The best case scenario with V3 is that it all just works; and the worst case is that it is entirely incompatible and TG becomes locked with V2. The likely case is that some things will pull through and others will result in conflicts of some sort.
For me the most important thing is stability and making sure that things continue to work in the next 20 years as FlightGear continues to evolve. Currently the codebase supports V1 and V2 scenery and this will probably continue so that V3 is also supported (I suspect there'd have to be a good reason to remove either of these). Unless someone backports V3 then older versions of FG will definitely not be able to use the new scenery. Backporting may be a non trivial task because of dependency changes that are necessary.
Hand crafted, detailed and often beautiful improvements to the scenery are fantastic to see and need to be encouraged; but in a way that will be compatible with the future. I'm not saying TG isn't compatible, as we can't know this, just that compatibility is an important consideration.
Pete is doing a making good progress; and it might just be one of those jobs that often are best done by a single mind. There is always a concern that if he gives up then V3 will not happen any time soon.
I'm still not well enough informed to be able to make a valued judgement about the future. I can see that there are possibilities with TG, but I can also see the risks in the longer term with compatibility and can't help wondering if what we should be looking for is a way to make changes to scenery in a compatible way so that we can write a script to go through git, figure out the differences and prepare these for submission.
It looks to me as though XPlane uses a similar source -> target database for submissions to the WED - and then builds out the apt.dat at timely intervals for their releases which definitely makes me think that the scenery database is probably a good idea that needs improvement rather than replacement.
Finally everything that is in the scenery should as close as possible to the real world and nothing fictional; this is manually reviewed for submissions to the scenery database so maybe there should be some way of change approval like in XPlane.