Customization of X-Plane10 is covered in the Installation instructions. This page describes how I created those customization files: airport layouts, nav data and maps, the base mesh, modified terrains, and bitmap versions of the base mesh.
Here is a summary plan for generating airports and scenery for Mars:
tools/WED
to create/update airports layouts.tools/apt.sh
to consolidate all individual apt.dat files.apts -l -b
, and the consolidated apt.dat, to create:apts -n -j
, and the consolidated apt.dat, to create:qgis
to generate a map of Mars, showing the location of all airports. [Optional]bitmapper
to generate files xxxx.dds
and
xxxx-nrm.png
, for rendering distant scenery.meshxes
, and some suitable climate profile, to generate 176x360 .xes files.meshhgt -vM
, and MOLA data, to generate 176x360 .hgt files. Flatten airports as well.makemesh.sh
to create 176x360 base tiles. For each tile:MeshTool tile.txt tile.xes tile.hgt borderData/ tile.dsf
tile.dsf
"Earth nav data/±n0±nn0/"
.ter
and .dds
files."Earth nav data/*"
to a subdirectory of X-Plane, and finally test.All these notes assume (but don't require) a Linux system, with about 200 GB of free disk space and at least 4 GB of RAM. I use GNU g++ for compiling the ~7000 lines of C code developed for this project.
The following steps involve a lot of modifications to core X-Plane
resource files. Consequently I recommend creating a separate copy of
X-Plane, to be customized for Mars: I put my copy in a directory called
M-Plane10/
.
Furthermore, the instructions below assume that the tool chain files
are installed in a sibling directory. This relationship makes
it simpler to transfer files between M-Plane10/
and
marsPackage/
.
|-- M-Plane10/ Contains clone of X-Plane10 | |-- Aircraft/ | |-- Global Scenery/ | |-- tools/ | : : | |-- Log.txt | : : |-- marsPackage/ Contains tool chain files | |-- bitmapper.c | |-- meshhgt.c | : :
As further preparation for what is to follow, you will need to
use 'make
' to compile the code, and ensure that the
MOLA dataset is present. The full MOLA dataset can be downloaded
as 16 simple 128 MB height files:
Download MOLA data by FTP to ./molaData/ for lat in 88n 44n 00n 44s; do for lon in 000 090 180 270; do wget pds-geosciences.wustl.edu/mgs/mgs-m-mola-5-megdr-l3-v1/mgsl_300x/meg128/megt${lat}${lon}hb.img wget pds-geosciences.wustl.edu/mgs/mgs-m-mola-5-megdr-l3-v1/mgsl_300x/meg128/megt${lat}${lon}hb.lbl Optional done; done
If you already have WED
then you can direct it to M-Plane10
by editing the
preference file, ~/.WED.prefs
. Otherwise, it can be
downloaded from Laminar:
cd M-Plane10 Contains clone of X-Plane10
mkdir tools
cd tools
wget https://developer.x-plane.com/tools/worldeditor/wed_lin_150r3.zip
unzip wed_lin_150r3.zip
Get MeshTool and associated XPTools:
cd marsPackage Contains tool chain files mkdir dump wget https://dev.x-plane.com/download/tools/meshtool_lin_300b1.zip wget https://dev.x-plane.com/download/tools/config_MT3.zip wget https://dev.x-plane.com/download/tools/xptools_lin_15-3.zip unzip meshtool_lin_300b1.zip MeshTool unzip config_MT3.zip config/*.txt (45 files) unzip xptools_lin_15-3.zip ObjView XGrinder tools/DDSTool tools/DSFTool rm -rf __MACOSX
The program versions indicated above are what I used. They date from 2015, and were compatible with all versions of X-Plane10. However don't feel bound to those specific versions. There are both newer and older versions of these programs that may be better in some way.
tools/WED
The airports in the package are created and maintained with the help of Laminar's World EDitor (WED). I used WED to ensure each airport has (a) a Tower Viewpoint, (b) a light beacon, and (c) an airport boundary. I also checked that elevations were correct, and that the runways were properly labelled. All these are essential for later steps.
Once the ICAO code and runway labels are settled, then suitable
entries need to be added to util/aptDat.cpp
. The entries
comprise ILS frequencies, glideslopes, and suchlike: information that
will be output to the navaid file (see step 4 below).
Example insertion into util/aptDat.cpp
{"MAAF:17", 11030, 25, "IMAE", "ILS-cat-I", 3.0},
{"MAAF:35", 11130, 25, "IMAF", "ILS-cat-I", 3.0},
The inherited airports sometimes had taxiways, sometimes not. I didn't add much to these airports' layouts, so they still look rather empty.
As explained in the Installation notes,
all the Martian airports should be installed under M-Plane10/Custom Scenery/
.
Then the program WED
should be downloaded from Laminar. It
doesn't matter where it is put: I installed it to M-Plane10/tools/
.
tools/apt.sh
The shell script apt.sh
consolidates all of the individual
apt.dat
files created by WED.
#! /bin/bash # Merges all files 'Custom Scenery/MA??/Earth nav data/apt.dat' # This script should be located in ~/M_Plane10/tools, and run thus: # tools/apt.sh # tools/apt.sh > Resources/'default scenery'/'default apt dat'/'Earth nav data'/apt.dat sceneryPacks='./Custom Scenery' earthNavData='Earth nav data' id=0 if [ ! -d "${sceneryPacks}" ]; then echo "ERROR: Run from M-Plane10/"; exit 1; fi echo "I" echo "1000 Generated by WorldEditor" echo ls -1 -d "${sceneryPacks}"/MA?? | while read marsDir; do let "id = id + 1" marsFile=${marsDir}/${earthNavData}/apt.dat echo tail -n +4 "${marsFile}" | grep -v '^99$' done echo "99"
The consolidated apt.dat
is used both as a Resource
file by X-Plane10, and as input to the next two steps in the tool chain.
The directory M-Plane10/Custom Scenery/
needs to contain
all airports in packages named "MAAB", "MACF", etc. Also, the script
apt.sh
should be located in cd M-Plane10/tools
.
cd M-Plane10
tools/apt.sh This just displays the consolidated data
tools/apt.sh > Resources/'default scenery'/'default apt dat'/'Earth nav data'/apt.dat
apts -l -b
The collective apt.dat
(created in the previous step) can
act as the basis for 'shape' files: files with a format commonly used in
spatial mapping. One of the jobs of the program apts
is to
generate these shape files.
Option -l
causes the program to generate shape files
containing 'point' information: the latitude/longitude of each airport,
along with the airport's ICAO code. See aptShapes/aptMars.*
.
These files are read by QGIS when creating maps.
Option -b
causes the program to generate shape files
containing 'polygon' information: the boundary of each airport. See
aptShapes/aptBoundary.*
. These files are used by
meshhgt
to define the extent of each airfield. [Airfields
are artificially levelled, to ensure that runways are usable.]
This program reads
M-Plane10/Resources/default scenery/default apt dat/Earth nav data/apt.dat
.
Type ./apts -h
to see the command line options.
cd marsPackage This folder must be a sibling to M-Plane10/. It contains all toolchain programs. ./apts -l -b Outputs aptShapes/aptMars.* and aptShapes/aptBoundary.*
apts -n -j
The collective apt.dat
can act as the basis for additional
data files. One of the jobs of the program apts
is to generate
these data files.
Option -n
causes the program to generate a file
earth_nav.dat
. Plausible positions are chosen for NDB, ILS,
GS, and DME antennas. Plausible names and frequencies are also assigned.
The output file should be copied to M-Plane10/Resources/default data/
Option -j
causes the program to generate a file
aptData.js
. This contains all airport and navaid data,
formatted as JavaScript. The advantage of this is that is can be
easily imported and rendered by an HTML document.
This program reads
M-Plane10/Resources/default scenery/default apt dat/Earth nav data/apt.dat
.
There is additional ILS data hard-coded in util/aptDat.cpp
;
so remember to update this file after adding or modifying runways in WED.
Type ./apts -h
to see the command line options.
./apts -n -j Outputs earth_nav.dat and aptData.js Move the two files to where they will be used mv earth_nav.dat ../M-Plane10/Resources/'default data'/ mv aptData.js docs/js/
QGIS is a GIS program with the ability to load and render raster data. In particular, it can load the 16 raw MOLA files, and colour the pixels according to elevation. Furthermore it can load the shape files generated earlier, so the names and locations of airports can be overlaid on the map.
Maps produced with the help of QGIS might be used as a navigational aid.
I put maps into the directory aptMapPDF/
. I tried labelling
mountains and craters, using the shape files in aptMapReproj/
.
However those shape files use some other coordinate system (0 to 360,
instead of -180 to +180) so it was not successful.
The MOLA files megtnnxnnnhb.img
need to be
downloaded, and placed within ./molaData/
, say. [Downloading
MOLA data was described at the top of this page.]
QGIS can also make use of the shape files produced earlier by apts -l
.
Generating a nice PDF map is easy with the configuration file
map.qgs
.
qgis map.qgs 1. Project / Print Composers / 'mars1' 2. Composer / Export as PDF / 'mars/aptMapPDF/mars.pdf' cp aptMapPDF/mars.pdf docs/images/
bitmapper [far | mid]
Apart from the base mesh and overlay DSFs there are two other
representations of the scenery. They have lower resolution, and are used
to render far distant scenery (latitude resolution ~0.2°),
and scenery in the middle distance (latitude resolution ~0.01°).
The data for each tile is contained in pairs of files,
xxxx.dds
and xxxx-nrm.png
.
The two members of each pair have complementary roles:
xxxx.dds
file contains four channels (RGBA):
xxxx-nrm.png
file is more interesting.
It too contains four channels (RGBA), however the channels have
unconventional roles:
128 + 127*norm_ew
.128 + 127*norm_ns
. [Trust me
on the direction. It is true, despite the fact that DSF
vertices have normals for which a positive NS component
means downhill southwards.]
XESCore/DEMIO.cpp
reveals that 'B'
is actually the vertical component of the surface normal.
Its potential range is [0,1], and is encoded as an 8-bit
integer 255*norm_v
. Why did I earlier reject
this obvious idea?255 - int[(418+elevInMetres)/(418+8848)*255]
.
The implicit range of allowed elevations, -418 – 8848m,
is not ideal for Mars. So much of Mars is above or below this
range! All we can do is truncate the scaled values, which
(unfortunately) creates artificial floors and ceilings.
The consequent visual artifacts are discussed below, in the
final section.Bitmaps are 'post-centric' arrays, so the nominal coordinate of each value is well-defined. Each value is the computed from a number of DEM values (since bitmaps are coarse representation of the DEM data) — but which DEM values, exactly? For a given pixel latitude and longitude, should the bitmap value be computed as an average of surrounding elevations, or values to north and east? It is hard to guess the workings of the X-Plane rendering engine. It is also hard to establish this empirically.
Some specs for these pairs of files:
Distance | # pairs | — Tile size — | Resolution | .dds files | -nrm.png files | |
---|---|---|---|---|---|---|
Far | 2 | 180°×180° | 1024×1024 | ~0.2° or 6.4nmi | earth{1,2}.dds | earth{1,2}-nrm.png |
Mid | 18×36 | 10°×10° | 1024×1024 | ~0.01° or 0.3nmi | ±n0±nn0.dds | ±n0±nn0-nrm.png |
Neighboring tiles overlap. The extreme right column of one tile is identical to column zero of its neighbor. Likewise for the top and bottom edges. This is known as 'post-centric', in the terminology of the DSF format
display filename.png Visual display xmag Magnifies, and shows RGB values identify -verbose filename.png Image info convert filename.png -alpha Deactivate -color-matrix "1.0 0.0 0.0 0.0 0.0 \ 1.0 0.0 0.0 0.0 0.0 \ 1.0 0.0 0.0 0.0 0.0 \ 0.0 0.0 0.0 0.0 0.0 \ 0.0 0.0 0.0 0.0 0.0" filename-R.png Extract R channel to 8-bit gray-scale convert filename.png -alpha Deactivate -color-matrix "0.0 1.0 0.0 0.0 0.0 \ 0.0 1.0 0.0 0.0 0.0 \ 0.0 1.0 0.0 0.0 0.0 \ 0.0 0.0 0.0 0.0 0.0 \ 0.0 0.0 0.0 0.0 0.0" filename-G.png Extract G channel to 8-bit gray-scale convert filename.png -alpha Deactivate -color-matrix "0.0 0.0 1.0 0.0 0.0 \ 0.0 0.0 1.0 0.0 0.0 \ 0.0 0.0 1.0 0.0 0.0 \ 0.0 0.0 0.0 0.0 0.0 \ 0.0 0.0 0.0 0.0 0.0" filename-B.png Extract B channel to 8-bit gray-scale convert filename.png -alpha Extract filename-A.png Extract A channel to 8-bit gray-scale convert filename.png txt:- Convert image to readable ascii # ImageMagick pixel enumeration: 1024,1024,255,srgba Row-wise enumeration, starting at top-left corner 0,0: (132,130,254,239) #8482FEEF srgba(132,130,254,0.937255) 1,0: (133,126,254,239) #857EFEEF srgba(133,126,254,0.937255) : : : : 1022,1023: (132,131,254,235) #8483FEEB srgba(132,131,254,0.921569) 1023,1023: (128,129,254,236) #8081FEEC srgba(128,129,254,0.92549) DDSTool --png2dxt5 dummy1 dummy2 in.png out.dds Convert from .png to .dds convert in.dds out.png Convert from .dds to .png
Bitmaps are represented in memory as long[][] arrays. Their general organization is:
unsigned int RGBA = ((A<<8 | B)<<8 | G)<<8 | R;
In my code, this array is passed to a function supplied by the Debian/Ubuntu package 'libpng12-dev', which encodes a .png file.
This is also what occurs in X-Plane's DSTools package. In
particular see
https://github.com/X-Plane/xptools/blob/master/src/XESCore/DEMIO.cpp
.
However there is a big source of confusion when comparing my code
and theirs: RGBA or BGRA? The PNG file format can handle both,
so either byte ordering can be supplied to the PNG library. XPTools
adopts the opposite convention to me, but one or other of us is
wrong in our labelling of the 'R' and 'B' bytes. This confusion is
acknowledged in the comment 'NOTES ON ENDIAN CHAOS' in
https://github.com/X-Plane/xptools/blob/master/src/Utils/BitmapUtils.cpp
.
Just to be clear: this muddle is only in the labelling in code used to create the bitmaps. The end result is fine.
X-Plane version 10.36 provided mars{1,2}.dds
, but none of the other
mid/far distant scenery for Mars. However we can generate our own, using MOLA
elevations. For simplicity we choose a uniform color (RGB=172,120,90).
The files generated by bitmapper
are:
far
'
docs/mars{1,2}.png
used as a figure in the documentationbitmaps/earth{1,2}.dds
for rendering Mars from a high orbitbitmaps/earth{1,2}-nrm.png
to complement the .dds imagesmid
'
bitmaps/Earth Orbit Textures/±n0±nn0.dds
bitmaps/Earth Orbit Textures/±n0±nn0-nrm.png
This program requires the MOLA elevation dataset
molaData/megt00n000hb.img, megt00n090hb.img, ...
It also
requires the utility program tools/DDSTool
, from Laminar's
XPTools package.
Type ./bitmapper -h
to see the command line options.
Then, typically:
./bitmapper far About 2 mins ./bitmapper mid About 20 mins rsync -av bitmaps ../M-Plane10/Resources Install
meshxes
This program generates a complete set of 176x360 .xes files, containing 'climate data' for Mars.
The .xes file format is not documented. However it is fairly easy to dissect a .xes file, and to generate suitable files from scratch.
The first part of each .xes file is a list of definitions. The remainder is a series of seven 1-sq deg maps of the quantities temp, temp range, rainfall, land use, soil type, agri style, clim style. The values are all f32, and are either physical quantities, or index items in the definitions list. Building a .xes file is just a matter of reproducing the definition list and appending seven maps containing plausible values.
To obtain the definitions is a usable form, take any .xes file, and do
Download an arbitrary land class file wget https://dev.x-plane.com/update/misc/MeshTool3/+20+120.zip unzip ./+20+120.zip First unzip unzip ./+20+120/+28+121.xes Second upzip Extract definitions strings Volumes/Hedge/MT3/+20+120/+28+121.xes > defs.txt vim defs.txt Eliminate a few lines of junk at the end gawk -f defs.awk defs.txt > xesDefs.h Create a C include file
The last line creates a C include file, which enables the defs
to be imported into meshxes
. It relies on an awk script,
defs.awk
:
BEGIN {n = 0; print "const char *defs[] = {"} {print " \"" $1 "\", // " n++} END {print "};\n#define nDefs (" n ")\n"}
A typical .xes file looks like this:
Offset Size Example value
000000 4+4 "TOKN" 0002f076
000008 2f06e NO_VALUE BARE FOREST ....... [6319 such 'definitions']
02f076 4+4 "MAP1" 00000054
02f07e 4+4 "MAP2" 0000004c
02f086 17x4 uint32 map2[17] = {0,0,1,1,1,0,0,0,1,0,0,0,0,0,0,0,0};
02f0ca 4+4 "MSH1" 00000080
02f0d2 4+4 "mesh" 00000024
02f0da 7x4 sint32 mesh[7] = {1,1,-1,0,0,0,0};
02f0f6 4+4 "dat1" 00000054
02f0fe 38x2 uint16 dat1[38] = {0,0,0,0,0,0,0,0, 0,0,0,0,0,0,0,0,
1,0,0,0x3ff0,0,0,0,0, 0,0,0x8000,0xffff,0,0,0,0,
0,0,0,0,0,0};
02f14a 4+4 "DEMd" 00000028
02f152 4 0000007
02f156 7x4 uint32 demd[7] = {9,10,11,25,27,28,29};
02f172 4x4 uint32 type = 9, size = 000e130, dim = 120, dim = 120
02f182 4x8 f64 lat0, lon0, lat1, lon1
02f1a2 120x120 f32 val[120x120] (e.g. dem_Temperature ~= 25.2 C)
03d2a2 4x4 uint32 type = 10, size = 000e130, dim = 120, dim = 120
03d2b2 4x8 f64 lat0, lon0, lat1, lon1
03d2d2 120x120 f32 val[120x120] (e.g. dem_TemperatureRange ~= 30.2 C)
04b3d2 4x4 uint32 type = 11, size = 000e130, dim = 120, dim = 120
04b3e2 4x8 f64 lat0, lon0, lat1, lon1
04b402 120x120 f32 val[120x120] (e.g. dem_Rainfall ~= 214.0 mm)
059502 4x4 uint32 type = 25, size = 0040a094 dim = 1029, dim = 1029
059512 4x8 f64 lat0, lon0, lat1, lon1
059532 1029x1029 f32 val[1029x1029] (e.g. dem_LandUse = 205.0 == lu_globcover_SPARSE)
463596 4x4 uint32 type = 27, size = 00003a54, dim = 61, dim = 61
4635a6 4x8 f64 lat0, lon0, lat1, lon1
4635c6 61x61 f32 val[61x61] (e.g. dem_SoilStyle = 4691.0 == SOIL_RED)
466fea 4x4 uint32 type = 28, size = 00000214, dim = 11, dim = 11
466ffa 4x8 f64 lat0, lon0, lat1, lon1
46701a 11x11 f32 val[11x11] (e.g. dem_AgriStyle = 0.0 == NO_VALUE)
4671fe 4x4 uint32 type = 29, size = 00000214, dim = 11, dim = 11
466ffa 4x8 f64 lat0, lon0, lat1, lon1
46701a 11x11 f32 val[11x11] (e.g. dem_ClimStyle = 0.0 == NO_VALUE)
467412 File size
The file structure and content is quite clear — apart from the three arrays identified as "MAP2", "mesh" and "dat1", whose meaning is still unclear to me. I have preserved these three arrays exactly as shown. However I felt free to modify the subsequent seven arrays — both their dimensions and the values therein.
In creating a set of .xes files, the goal is to specify climate-related
quantities that (when combined with config/terrain_rules.txt
)
causes MeshTool to generate suitable looking terrain. The choices tabulated
below describe very dry land, but with a temperature profile resulting in
patchy ice at extreme latitudes.
Climate component | Meaning and content |
---|---|
9:dem_Temperature | Average temperature in Celsius. I specify a parabolic
profile, val = 30 - 50*(lat/90.0f)*(lat/90.0f) |
10:dem_TemperatureRange | Temperature range in Celsius. I don't think this has much effect. I specified a fixed value of 31C. |
11:dem_Rainfall | Average rainfall in mm/year (possibly). I specified a value of 10 mm/year, which ensures that MeshTool chooses the driest looking terrain. |
25:dem_LandUse | Land usage. This quantity should be one of those
listed in the definitions section, of which
seem most pertinent. Unfortunately the list offers nothing
resembling 'dustbowl'. The rules appearing in
|
27:dem_SoilStyle | Soil type. I specify
val = 4691; // "SOIL_RED" planet-wide.
This is clearly OK for much of the planet, although extreme
latitudes nevertheless are represented as bare icy rock. |
28:dem_AgriStyle | Agricultural type. I specify
val = 0; // "NO_VALUE" planet-wide. |
29:dem_ClimStyle | Climate type. I specify
val = 0; // "NO_VALUE" planet-wide. |
MeshTool uses these .xes files later. MeshTool will
use values in these files to help choose appropriate .ter
terrains. The rules are in the file
config/terrain_rules.txt
, which is structured as follows:
Col: 0 1 2 3 4 5 6 ------------------------------------------------------------------------------------------------------------- #TERRAIN_RULE TERRAIN ZONING LAND USE SOIL_STYLE AGRI_STYLE CLIM_STYLE # ------------------------------------------------------------------------------------------------------------- TERRAIN_RULE terrain_Airport NO_VALUE NO_VALUE NO_VALUE NO_VALUE NO_VALUE TERRAIN_RULE NO_VALUE NO_VALUE *various* NO_VALUE NO_VALUE CLIM_TUNDRA TERRAIN_RULE NO_VALUE NO_VALUE *various* NO_VALUE NO_VALUE CLIM_TUNDRA : : : : : : : Col:7 8 9 10 11 12 13 14 15 16 17 ----------------------------------------------------------------------------------------- ELEVATION SLOPE TEMP TEMP RANGE RAINFALL WATER MIN MAX MIN MAX MIN MAX MIN MAX MIN MAX PROX ----------------------------------------------------------------------------------------- 0 0 0 0 -999 -8.0 0 0 0 55 0 0 0 8 19 -999 -8.0 0 0 0 55 0 0 0 0 8 -999 -8.0 0 0 0 55 0 : : : : : : : : : : : Col:18 19 20 21 22 23 24 ------------------------------------------------------------------------ REL ELEV ELEV RANGE LATITUDE NAME MIN MAX MIN MAX MIN MAX ------------------------------------------------------------------------ 0 0 0 0 0 80 terrain10/apt_vpol_dry 0 0.1 0 650 0 80 terrain10/tun_spr_pol_dry_sflat 0 0.1 0 650 0 80 terrain10/tun_spr_pol_dry_fl : : : : : : : Note: 1 In the above '*various*' stands for: lu_globcover_BARE_ROCKS,lu_globcover_BARE_CONSOLIDATED,lu_globcover_BARE_SCREE,lu_globcover_SALT, lu_globcover_SALT_HEM,lu_globcover_MORAINE,lu_globcover_MORAINE,lu_globcover_MINE 2 Values 0 and NO_VALUE generally mean 'ignore'. 3 Mars data extends to 88 degrees of latitude. To get terrain_rules.txt to work at extreme latitudes it is necessary to modify column 23, replacing '80' with '88'.
The same data is provided in a .ods spreadsheet. Here is a guide to this spreadsheet. The last column shows the sources of values required by the algorithm.
Column(s) | in .ods | Name | Value comes from |
---|---|---|---|
1 | B | TERRAIN | See script file (i.e. BACKGROUND and SHAPEFILE_TERRAIN commands) |
2 | C | ZONING | NA: the entry NO_VALUE probably means that it is ignored |
3 | D | LAND USE | See section 25, 'dem_LandUse', in .xes file |
4 | E | SOIL_STYLE | See section 27, 'dem_SoilStyle', in .xes file |
5 | F | AGRI_STYLE | See section 28, 'dem_AgriStyle', in .xes file |
6 | G | CLIM_STYLE | See section 29, 'dem_ClimStyle', in .xes file |
7-10 | H-K | ELEV, SLOPE | Inferred from .hgt DEM file |
11-12 | L-M | TEMP | See section 9, 'dem_Temperature', in .xes file |
13-14 | N-O | TEMP RANGE | See section 10, 'dem_Temperature', in .xes file |
15-16 | P-Q | RAINFALL | See section 11, 'dem_Rainfall', in .xes file |
17 | R | WATER PROX | NA: this entry is always 0 |
18-19 | S-T | REL ELEV | This value (0.0 .. 1.0) is inferred from .hgt file |
20-21 | U-V | ELEV RANGE | This value (in metres?) is directly from the .hgt file |
22-23 | W-X | LATITUDE | This value can be inferred from the file names |
24 | Y | NAME | This is the chosen .ter file |
The first rule that matches is the one that determines the .ter file for that point in the mesh. There are fall-back rules at the end that ensure that some choice is always made. The names of .ter files in the last column are informative; most are obvious, but note: pol=polar, vpol=very polar, sdry=semi-dry, lo/md/tp=low/middle/top of hill, fl=flat, sflat=slight slope, sq=square blocks, morn=moraine, sp=sparse, spr=sparse rock.
How MeshTool uses .xes files
The bottom line is that the .xes
specifications
(plus .hgt
values) will help MeshTool to choose the
semi-plausible terrain files. (BTW, terrain files are located in
Resources/default scenery/1000 world terrain/terrain10/
.)
The list below is not quite complete, but you get the picture: there
is a good spread of terrains selected by MeshTool, Nevertheless, they
are standard X-Plane terrains, originally intended for Earth. It
would of course be preferable to create terrains that are customized
for Mars, however I am not a graphic artist, a geographer, nor Mars
geology specialist. Anyone?
At most latitudes lib/g10/terrain10/red_spr_{hot|vhot}_dry_sflat Rendered as sand lib/g10/terrain10/red_spr_{hot|vhot}_dry_fl Rendered as sand lib/g10/terrain10/red_{rock|sand}_{hot|vhot}_dry_steep.ter lib/g10/terrain10/red_{rock|sand}_{hot|vhot}_dry_hill.ter lib/g10/terrain10/red_{rock|sand}_{hot|vhot}_dry_sflat.ter lib/g10/terrain10/red_{rock|sand}_{hot|vhot}_dry_fl_md.ter lib/g10/terrain10/red_{rock|sand}_{hot|vhot}_dry_fl_tp.ter lib/g10/terrain10/red_{rock|sand}_{hot|vhot}_dry_fl.ter lib/g10/terrain10/apt_{hot|vhot}_dry.ter At polar latitudes lib/g10/terrain10/{rock|sand}_{vpol|vcld}_dry_steep.ter lib/g10/terrain10/{rock|sand}_{vpol|vcld}_dry_hill.ter lib/g10/terrain10/{rock|sand}_{vpol|vcld}_dry_sflat.ter lib/g10/terrain10/{rock|sand}_{vpol|vcld}_dry_fl_md.ter lib/g10/terrain10/{rock|sand}_{vpol|vcld}_dry_fl_tp.ter lib/g10/terrain10/{rock|sand}_{vpol|vcld}_dry_fl.ter lib/g10/terrain10/snow_{vpol|vcld}_dry_flat.ter lib/g10/terrain10/snow_{vpol|vcld}_dry_flat_lo.ter lib/g10/terrain10/snow_{vpol|vcld}_dry_sflat.ter lib/g10/terrain10/snow_{vpol|vcld}_dry_sflat_lo.ter lib/g10/terrain10/snow_{vpol|vcld}_dry_hill.ter lib/g10/terrain10/snow_{vpol|vcld}_dry_steep.ter lib/g10/terrain10/apt_{vpol|vcld}_dry.ter
This program requires that landType
has already been run,
so that the directory TerrainAssignments/
is fully populated.
The program landType
digests geological land class data
for Mars, obtainable from USGS.
It reads polygon shape files, rasterizes the data, and writes 176x360 1x1
degree maps to files under TerrainAssignments/
. This geological
data is referred to by meshxes
.
Type ./meshxes -h
to see the command line options.
./meshxes About 2 hours
meshhgt -vM
This program reads MOLA elevation data (the 16 .img files in
molaData/
) and outputs 63360 .hgt files. Each file
covers 1x1 degree, and has a format suitable for use with
MeshTool. The hierarchical directory structure of the output
matches that of other scenery files:
"hgt/Earth nav data/±n0±nn0/±nn±nnn.hgt
".
The raw MOLA elevations are interpolated – from 128 points
per degree to 300 points per degree – to help MeshTool introduce
more spatial variations. This level of interpolation ultimately leads
to 17 GB of .dsf
files. Finer interpolation results in
nicer looking scenery, but then downloading becomes a hurdle.
This program also performs flattening within each airport's boundary: natural height variations around the mean are reduced to 3%. This is intended to make the runway flat enough to use, but not unnaturally flat.
A minor additional job for this program is to build a replacement for
Resources/dsf_files
. This resource file is 64805 bytes
long, comprising: the character 'i', the number 860 as a little-endian
4-byte integer, and 180×360 bytes (0x00 for empty sea, 0x01 implies
a DSF tile is available). It can be inspected with
tail -c +6 dsf_files | tr '\000\001' ' #' | fold -b -w 180
.
Row 0 contains longitude -180, row 1 contains longitude -179, etc; and
latitudes run from -90 (leftmost) to +89 (rightmost).
This program requires
molaData/megt00n000hb.img, megt00n090hb.img,
...aptShapes/aptBoundary.{shp,shx,dbf}
Type ./meshhgt -h
to see the command line options.
The option -vM
is recommended.
./meshhgt -vM About 3 mins
mv dsf_files ../M-Plane10/Resources/
makemesh.sh
MeshTool requires a script file, containing supplementary data.
The command MESH_SPECS max_added_points max_error
prevents MeshTool getting carried away trying to match neighboring
tiles, and conforming to the supplied DEM. The command BACKGROUND
base_terrain
means that terrain is generated
automatically (with the help of config/terrain_rules.txt
).
The command SHAPEFILE_TERRAIN terrain_Airport aptShapes/aptBoundary.shp
means that suitable airport terrains will be 'burnt into' the base mesh.
This file will serve for all 63360 tiles:
MESH_SPECS 78000 6.5 BACKGROUND terrain_Natural SHAPEFILE_TERRAIN terrain_Airport aptShapes/aptBoundary.shp
MeshTool is quite easy to run, once all the preparatory work is done.
However the shell script makemesh.sh
is handy because:
hgt/Earth nav data/±n0±nn0/±nn±nnn.hgt
,
so are tedious to type, and tricky to get right7zr
)makemesh.sh
[Click to show/hide]
#! /bin/bash # This runs MeshTool multiple time, to create a full set # of .dsf files. Presently these files are not compressed. # This script accepts 0, 2 or 4 command line arguments # makemesh.sh -88 to +87 -180 to +179 # makemesh.sh lat0 lat1 lat0 to lat1 -180 to +179 # makemesh.sh lat0 lat1 lon0 lon1 # E.g. # ./makemesh.sh -80 -61 # ./makemesh.sh -20 -11 -60 -51 # We assume that .xes and .hgt files have already been # created, for example by # ./meshxes -vM -dR # ./meshhgt -vM # Enable Ctrl-C; cursor must be in the terminal window trap exit 2 15 lat0=-88 lat1=87 lon0=-180 lon1=179 if [ $# -eq 0 ]; then # Process the full range of lats and lons # However runtime would be unreasonably long, so exit echo "Generating the full 176x360 dataset will take ~10 days!" echo "Type './makemesh.sh -h' to see more sensible options" exit 0; elif [ $# -eq 2 ]; then lat0=${1} lat1=${2} elif [ $# -eq 4 ]; then lat0=${1} lat1=${2} lon0=${3} lon1=${4} else echo "Usage:" echo " ${0} Generate -88 to 87 & -180 to 179 DISABLED!" echo " ${0} lat0 lat1 Generate lat0 to lat1 & -180 to 179" echo " ${0} lat0 lat1 lon0 lon1 Generate lat0 to lat1 & lon0 to lon1" exit 2 fi # Simple validity checks echo "Latitudes: " $lat0 "to" $lat1 echo "Longitudes: " $lon0 "to" $lon1 if [ $lat0 -gt $lat1 ]; then echo "ERROR: Latitudes need to be swapped!" exit 2 elif [ $lon0 -gt $lon1 ]; then echo "ERROR: Longitudes need to be swapped!" exit 2 fi if [ $lat0 -lt -88 -o $lat0 -gt 87 -o $lat1 -lt -88 -o $lat1 -gt 87 ]; then echo "ERROR: Latitudes need to be in the range -88 to 87!" exit 2 elif [ $lon0 -lt -180 -o $lon0 -gt 179 -o $lon1 -lt -180 -o $lon1 -gt 179 ]; then echo "ERROR: Longitudes need to be in the range -180 to 179!" exit 2 fi # Run MeshTool for((lat=$lat0; lat<=$lat1; lat++)); do for((lon=$lon0; lon<=$lon1; lon++)); do let "latR=((lat+90)/10*10)-90" let "lonR=((lon+180)/10*10)-180" latRF=$(gawk '{printf "%+03d\n", $0;}' <<< $latR) lonRF=$(gawk '{printf "%+04d\n", $0;}' <<< $lonR) latF=$(gawk '{printf "%+03d\n", $0;}' <<< $lat) lonF=$(gawk '{printf "%+04d\n", $0;}' <<< $lon) # Create numeric directory and file names d1=${latRF}${lonRF} fn=${latF}${lonF} # Create command line arguments dir="Earth nav data/"${d1} xes="xes/"${d1}/${fn}.xes hgt="hgt/"${d1}/${fn}.hgt out1="./"${fn}.dsf out2="Earth nav data/"${d1}/${fn}.dsf echo echo echo "-----------------------------------------------------------" echo ./MeshTool meshscript.txt $xes $hgt dump $out1 echo "-----------------------------------------------------------" mkdir -p "$dir" ./MeshTool meshscript.txt $xes $hgt dump $out1 if [ ! -e $out1 ]; then exit 1; fi rm -f "$out2" 7zr a -mx=9 -mfb=64 -md=32m "$out2" $out1 rm $out1 done done
This script requires
meshscript.txt
xes/*/*.xes
hgt/*/*.hgt
It also makes use of files in ./config/
and
./aptShapes/
. N.B. A crucial modification
needs to be made to ./config/terrain_rules.txt
. By
default, rules are limited to latitudes within ±80°, and
MeshTool will crash for more extreme latitudes. However the fix is
simple: replace '80' with '88' in column 23. [One way to do this is
sed 's/80\tterrain10/88\tterrain10/' terrain_rules.txt
]
Type ./makemesh -h
to see the command line options.
The script accepts command line arguments, which enables it to generate
only a specific subset of the full 63360 set.
./makemesh.sh lat0 lat1 Do only latitudes lat0 to lat1 (inclusive); longitudes -180 to 179 are implied ./makemesh.sh lat0 lat1 lon0 lon1 Do only latitudes lat0 to lat1, and longitudes lon0 to lon1 (inclusive) ./makemesh.sh 4 4 -38 -38 Example of generation of a single .dsf tile, whose SW corner is at lat=4, lon=-38
Latitude arguments must be integers in the range -88 to 87 (inclusive). Longitude arguments must be integers in the range -180 to 179 (inclusive).
It is possible to use the shell script to generate all 63360 .dsf files
— which will take about 10 days. However the smarter thing to do is
to run several instances of makemesh.sh
simultaneously, to make
best use of multiple cores. The commands for 6-way parallel construction of
.dsf files are something like:
This is sometimes necessary, and harmless at other times rm -rf dump/* Execute these commands in different term windows ./makemesh.sh -88 -58 ./makemesh.sh -57 -29 ./makemesh.sh -28 -1 ./makemesh.sh 0 27 ./makemesh.sh 28 56 ./makemesh.sh 57 87
With six processes running in parallel the total runtime is about 48 hours.
7zr e -o/tmp ./"Earth nav data"/-20-060/-12-059.dsf ./parseDsf -vv /tmp/-12-059.dsf | grep terrain10 | sort
.ter
and .dds
filesThe 176x360 .xes
files (see above) each consist of coarse
maps of temperature, rainfall, land usage, etc. Meshtool combines these
with config/terrain_rules.txt
to choose textures for each
triangle of the base mesh.
The file config/terrain_rules.txt
contains pointers to
.ter
files (meta texture files), which themselves contain
pointers to .dds
files (actual texture files). Both should
to be modified to create more Mars-like terrain textures.
Minor edits to .ter
files
The stage involving meshxes
specified a red, dry, rocky
terrain for all tiles. This goes a long way to creating Mars-like
scenery. We can further improve the appearance of the terrain by
converting flat areas to sand dunes, mimicking the dusty dunes
of Mars. Another job is to replace 'apt_tmp_dry
', since
that terrain is far too green.
The following recipe involves renaming .ter
.
I didn't investigate editing .ter
files, although that
might be a good course, if I had .png
images for Mars, or
if I wanted to use the alternative terrain files in
Resources/'default scenery'/'1000 world terrain'/pol9/
.
Here is how to make the necessary changes to selected .ter
files:
cd ~/M-Plane10/ tex=Resources/'default scenery'/'1000 world terrain'/terrain10 Dry flat 'spr' terrains → sand for zone in pol vcld cld; do ln -sf sand_${zone}_dry_fl.ter "${tex}"/spr_${zone}_dry_fl.ter ln -sf sand_${zone}_dry_sflat.ter "${tex}"/spr_${zone}_dry_sflat.ter done for zone in tmp wrm hot vhot; do ln -sf red_sand_${zone}_dry_fl.ter "${tex}"/red_spr_${zone}_dry_fl.ter ln -sf red_sand_${zone}_dry_sflat.ter "${tex}"/red_spr_${zone}_dry_sflat.ter done Temperate airports → warm ln -sf apt_wrm_dry.ter "${tex}"/apt_tmp_dry.ter
Resources/default scenery/1000 world terrain/library.txt
Minor edits to .dds
files
By default, sand is rather white in appearance. To make it more Mars-like we can redden the .dds images used for rendering sand. One way is as follows.
Use convert, described in https://www.imagemagick.org/script/command-line-options.php mkdir -p dds/soil New directory, to hold working copy of updated textures tex=~/M-Plane10/Resources/'default scenery'/'1000 world terrain'/textures10 for f in "${tex}"/soil/sand_*.dds; do pathTail=${f##*/} convert -modulate 80,140,95 "$f" /tmp/tex.png Alter brightness,saturation,hue tools/DDSTool --png2dxt5 dummy1 dummy2 /tmp/tex.png dds/soil/${pathTail} .png --> .dds echo "Updated ${pathTail}" done Install updated textures cp dds/soil/*.dds "${tex}"/soil/
Another minor edit is to eliminate tinges of green from four images in
Resources/default scenery/1000 world terrain/textures10/soil/
,
namely
red_rock_wrm_dry_sflat_d.dds
,
red_rock_wrm_dry_steep_d.dds
,
red_rock_wrm_dry_hill_d.dds
, and
red_rock_tmp_dry_hill_d.dds
. Many terrain files
reference these images, so the suggestion of green bushes therein
is quite distracting. However we can use
ImageMagick
to create a mask, then modulate greens into browns.
tex=~/M-Plane10/Resources/'default scenery'/'1000 world terrain'/textures10 for f in "${tex}"/soil/red_rock_*.dds; do 1. Identify pixels with both green hue and good saturation convert "$f" -colorspace HSL -channel Hue,Saturation -separate +channel \ \( -clone 0 -background none -fuzz 10% +transparent gray20 \) \ select hue=20 ± 10% \( -clone 1 -background none -fuzz 10% -transparent gray20 \) \ select all except saturation=20 ± 10% -delete 0,1 -alpha extract -compose multiply -composite -negate \ /tmp/mask.png 2. Make these pixels 50% brighter, 50% less vivid, and rotate hues by -25*(180/100)°: green → brown convert "$f" -mask /tmp/mask.png \ -modulate 150,50,75 +mask \ modulate brightness,saturation,hue /tmp/tex.png 3. Compress tools/DDSTool --png2dxt5 dummy1 dummy2 /tmp/tex.png dds/soil/${f##*/} .png --> .dds done Install updated textures cp dds/soil/*.dds "${tex}"/soil/
We also need to redden the texture that is used for 'cld', 'vcld',
'pol' and 'vpol' airport terrains. The terrain files
apt_*dry.ter
all reference one texture file
../textures10/apt/apt_cld_dry.dds
. So, similarly to the above cases:
tex=~/M-Plane10/Resources/'default scenery'/'1000 world terrain'/textures10
convert -modulate 100,80,90 "${tex}"/apt/apt_cld_dry.dds /tmp/tex.png
tools/DDSTool --png2dxt5 dummy1 dummy2 /tmp/tex.png dds/apt/apt_cld_dry.dds
Install updated texture
cp dds/apt/*.dds "${tex}"/apt/
To test the new base mesh, copy the contents of Earth nav data/
to a suitable X-Plane directory. I prefer the first of the two alternatives below:
rsync -av "Earth nav data" "../M-Plane10/Global Scenery/X-Plane 10 Global Scenery" The effect is to overwrite the base mesh. RECOMMENDED OR rsync -av "Earth nav data" "../M-Plane10/Custom Scenery/Trial-yyyymmdd" The effect is to override the base mesh with a new custom package
A further step may be to bundle all scenery as zip files, for easier distribution. There are makefile commands for this:
Assemble files in ./distro/
make docs.zip 3 MB
make mars.zip Makes 9 files mars[1-9].zip
, totalling 17 GB
make bitmaps.zip 680 MB
make ter.zip 6 kB
make dds.zip 80 MB
make apts.zip 220 kB
make other 280 kB 6 files: apt.dat earth_nav.dat dsf_files sky_colors_mars.png sun_glow_mars.png README
Here is the complete environment for building Mars scenery from scratch:
|-- aptMapPDF/ | |-- various maps derived with QGIS | : : |-- aptMapReproj/ | |-- various extra shape files containing labels for maps | : : |-- aptShapes/ | |-- various shape files derived from apt.dat | : : |-- archive/ | |-- various | : : |-- config/ from Laminar | |-- absolute_terrain.txt | |-- terrain_rules.txt | : : |-- docs/ | |-- index.html | |-- toolchain.html | : : |-- dump/ workspace for MeshTool |-- md5/ | |-- makefile | |-- md5 | |-- md5main.cpp | |-- md5main.o | |-- Md5Sum.cpp | |-- Md5Sum.h | `-- Md5Sum.o |-- tools/ from Laminar | |-- DDSTool | `-- DSFTool |-- shapelib-1.3.0/ | |-- various utility code | : : |-- I1802ABC_Mars_global_geology/ | |-- various geological data files needed by landtype | : : |-- SIM3292_MarsGlobalGeologicGIS_20M/ | |-- various geological data files needed by landtype | : : |-- wms2pol_1.56/ http://www.cestomano.com/mas/xplane/ | |-- wms2pol.py Generates draped orthophotos | `-- wms2pol.ini |-- dds/ | |-- apt/ | | `-- apt_cld_dry.dds just this files | `-- soil/ | |-- red_rock_*.dds 4 such files | `-- sand_*.dds 15 such files |-- molaData/ | |-- megt00n000hb.img 16 such files | `-- megt00n000hb.lbl 16 such files |-- bitmaps/ | |-- earth{1,2}.dds | |-- earth{1,2}-nrm.png | `-- Earth Orbit Textures/ | |-- ±n0±nn0.dds 18x36 such files | `-- ±n0±nn0-nrm.png 18x36 such files |-- TerrainAssignments/ | `-- ±nn±nnn/ 18x36 such directories | `-- ±nn±nnn.dat 100 such files |-- hgt/ | `-- ±nn±nnn/ 18x36 such directories | `-- ±nn±nnn.hgt 100 such files |-- xes/ | `-- ±nn±nnn/ 18x36 such directories | `-- ±nn±nnn.xes 100 such files |-- Earth nav data/ | `-- ±nn±nnn/ 18x36 such directories | `-- ±nn±nnn.dsf 100 such files |-- util/ | |-- aptDat.cpp | |-- aptDat.h | |-- aptDat.cpp | |-- flatten.c | |-- hel.cpp | |-- hel.h | |-- landTypeEncodings.c | |-- landTypeEncodings.h | |-- makefile | |-- mkdirs.c | |-- readMola.c | |-- readMola2.c | |-- rwy.cpp | |-- rwy.h | |-- writeDsf2.c | |-- writeDsf2.o | |-- writeXes.c | |-- writeXes.o | `-- xesDefs.h |-- apts #3 and #4 in tool chain |-- apts.cpp |-- bitmapper #6 in tool chain |-- bitmapper.c |-- landType [Companion to marsmesh] |-- landType.c |-- makefile |-- meshscript.txt |-- makemesh.sh #9 in tool chain |-- map.qgs |-- marsmesh [Creates simple mesh in one step] |-- marsmesh.c |-- meshhgt #8 in tool chain |-- meshhgt.c |-- meshxes #7 in tool chain |-- meshxes.c |-- parseDsf [Diagnostic] |-- parseDsf.c |-- MeshTool from Laminar |-- ObjView from Laminar `-- XGrinder from Laminar
Disk usage and runtime is something to be aware of.
Command | File | Disk usage | Runtime |
---|---|---|---|
bitmapper mid | .png | 648x2.3MB = 1500MB | 648x2s = 20 mins |
meshxes | .xes | 63360x20kB = 2GB | 63360x0.1s = 2 hours |
meshhgt | .hgt | 63360x1MB = 11GB | 63360x0.003s = 3 mins (if resolution is 301x301) |
makemesh.sh | .dsf | 63360x0.3MB = 17GB | 63360/Nx19s = 14/N days (N is the number of cores used) |
make mars.zip | .zip | 9x1.8GB = 17GB | 9x120s = 18 min |
Here are some limitations. Some are limitations to the Mars packages, and others are limitations inherent to X-Plane10. In no particular order:
MeshTool is clever at varying terrain based on topography and climate. However on it own it doesn't entirely solve the quilt-like effect of repeated tiling. Also the transition between different terrains is too abrupt. I guess both these issues can be addressed by using overlay DSFs.
A related problem is with perception of scale. Mountains can be huge on Mars, but — in the absence of medium scale landmarks for reference — they really don't look very impressive. It is a big problem when flying visually, since you can't judge how far and high mountains are. Again, I guess we need overlay DSFs.
As explained above in the section dealing with
meshxes
, I went along with the pre-existing
configuration files used by MeshTool
. I specified a
dry climate, red soil and rocks; and left it to MeshTool
to choose .ter
texture files. Tweaks were made to some
.ter
and .dds
files. Nevertheless,
the results may not be as accurate as they could be, and I don't
know enough about Martian geology to do any better. As pointed
out by Michael Gilliat, orthophotos are the answer, once higher
resolution images become available.
There is a big problem with middle distance scenery.
Reasonable blending occurs between DSFs and the mid-distance
scenery. However mid-distance elevations are encoded in the
'A' channel of the pseudo images
Earth Orbit Textures/±n0±nn0-nrm.png
,
and are limited to elevations in the range -418 to 8848m.
Unfortunately half of Mars is below the standard datum, so the
middle distance is missing for much of Mars. There is a hazy
void in the distance, which is suddenly filled in as the plane's
progress causes the next DSF tile to load (or when the renderer
falls back to using the far-scenery).
While this is an annoying and obvious artifact, it can be worse. In some situations there is a ghostly surface floating in the sky at elevation = -418m, while the ground elevation is far below. There is no real solution to this, with X-Plane as it is. A partial solution to floating translucent surfaces is to uncheck the
option in the menu . This turns off the mid-distance scenery.The middle distance scenery has realistic elevations and
shadowing, but has a uniform hue — generic Mars red-brown.
To do any better would involve considerable effort, as it
would be necessary to digest all .dsf
files, and
all the .ter
and .dds
files referenced
therein.
Mars has no magnetic field, so we would like magnetic compasses
to point to true north, rather than to (Earth's!) magnetic north.
Ideally we would customize X-Plane's model for magnetic
declination/variation. Unfortunately Earth's field seems to be
hard-coded within X-Plane10, precluding any simple tweaks (this
topic has been raised in the forum), nor is the dataref
sim/flightmodel/position/magnetic_variation
writeable.
However some instruments have writeable heading datarefs, so perhaps
a plugin can implement a fix, of sorts. Meanwhile, if you want to
fly a particular heading, then you will have to add (Earth's)
magnetic declination to your desired true heading. [You can work out
local declination by going to
,
and enabling item 19, 'magnetic compass'.]
Acknowledgements |