[ { "path": ".gitignore", "content": "# Python-related files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# Training data files.\nzip_files/\narray_files/\ntraining_images/\n*.csv\n*.pkl\n\n# Generated terrain files.\nml_outputs/\nsim_snaps/\n*.np[yz]\n\n# Misc files.\n.DS_Store\n*.swp\n" }, { "path": "LICENSE.txt", "content": "MIT License\n\nCopyright (c) 2018 Daniel Andrino\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "README.md", "content": "# Three Ways of Generating Terrain with Erosion Features\n\n## Background\n\nTerrain generation has long been a popular topic in the procedural generation community, with applications in video games and movies. Some games use procedural terrain to generate novel environments on the fly for the player to explore. Others use procedural terrain as a tool for artists to use when crafting a believable world. \n\nThe most common way of representing array is a 2D grid of height values. This type of terrain doesn't allow for overhangs and caves, but at large scales those features are not very apparent. The most popular terrain generation algorithms focus on adding together different layers of [coherent noise](http://libnoise.sourceforge.net/coherentnoise/index.html), which can be thought of as smoothed random noise. Several popular choices for coherent noise are:\n\n* [**Perlin noise**](https://en.wikipedia.org/wiki/Perlin_noise) - A form of [gradient noise](https://en.wikipedia.org/wiki/Gradient_noise) on a rectangular lattice.\n* [**Simplex noise**](https://en.wikipedia.org/wiki/Simplex_noise) - Like Perlin noise, but on a simplex lattice.\n* [**Value noise**](https://en.wikipedia.org/wiki/Value_noise) - Basically just white noise that's been upscaled and interpolated.\n\nIf you take several layers of coherent noise, each at different levels of detail and with different amplitudes, you get a rough pattern frequently (and mostly inaccurately) called [**fBm**](https://en.wikipedia.org/wiki/Brownian_surface) (fractional Brownian motion). [This page](https://www.redblobgames.com/maps/terrain-from-noise/) provides a good overview for how this process works. \n\nIn addition, there are other methods of generating fBm more directly, including:\n\n* [**Diamond-square**](https://en.wikipedia.org/wiki/Diamond-square_algorithm) - A fast, but artifact-prone divide-and-conquer approach.\n* **Power-law noise** - Created by filtering white noise in the frequency domain with a power-law function.\n\nWhat you get from regular fBm terrain is something like this:\n\n

\n \n \n
\n Typical fBm-based terrain. Left is height as grayscale, right is with coloration and hillshading. \n

\n\n\nThis gives reasonable looking terrain at a quick glance. It generates distinguishable mountains and valleys, and has a general roughness one expects from rocky terrain.\n\nHowever, it is also fairly boring. The fractal nature of fBm means everything more or less looks the same. Once you've seen one patch of land, you've basically seen it all. \n\nOne method of adding a more organic look to terrain is to perform [domain warping](http://www.iquilezles.org/www/articles/warp/warp.htm), which is where you take regular fBm noise but offset each point by another fBm noise map. What you get is terrain that looks warped and twisted, somewhat resembling terrain that has been deformed by tectonic movement. The game No Man's Sky uses domain warping for its custom noise function called [uber noise](https://youtu.be/SePDzis8HqY?t=1547).\n\n

\n \n \n
fBm with domain warping \n

\n\nAnother way of spicing up fBm is to modify each coherent noise layer before adding them together. For instance, if you take the absolute value of each coherent noise layer, and invert the final result you can get a mountain ridge effect: \n\n

\n \n \n
Modified fBm to create mountain ridges. \n

\n\nThese all look iteratively more convincing. However, if you look at actual elevation maps, you will notice that these look nothing like real life terrain:\n\n

\n \n \n \n \n\n
Elevation maps from somewhere in the continental United States (credit to the USGS). The right images uses the same coloration as above, for consistency. \n

\n\nThe fractal shapes you see in real life terrain are driven by **erosion**: the set of processes that describe terrain displacement over time. There are several types of erosion, but the one that most significantly causes those fractal shapes you see is **hydraulic erosion**, which is basically the process of terrain displacement via water. As water flows across terrain, it takes sediment with it and deposits it downhill. This has the effect of carving out mountains and creating smooth valleys. The fractal pattern emerges from smaller streams merging into larger streams and rivers as they flow downhill.\n\nUnfortunately, more involved techniques are required to generate terrain with convincing erosion patterns. The following three sections will go over three distinct methods of generating eroded terrain. Each method has their pros and cons, so take that into consideration if you want to include them in your terrain project.\n\n\n## Simulation\n\nIf real life erosion is driven by physical processes, couldn't we just simulate those processes to generate terrain with erosion? Then answer is, yes! The mechanics of hydraulic erosion, in particular, are well known and are fairly easily to simulate.\n\nThe basic idea of hydraulic erosion is that water dissolves terrain into sediment, which is then transported downhill and deposited. Programmatically, this means tracking the following quantities:\n\n* **Terrain height** - The rock layer that we're interested in.\n* **Water level** - How much water is at each grid point.\n* **Sediment level** - The amount of sediment suspended in water.\n\nWhen simulating, we make small changes to these quantities repeatedly until the erosion features emerge in our terrain.\n\nTo start off, we initiate the water and sediment levels to zero. The initial terrain height is seeded to some prior height map, frequently just regular fBm.\n\nEach simulation iteration involves the following steps:\n\n1. **Increment the water level** (as in via precipitation). For this I used a simple uniform random distribution, although some approaches use individual water \"droplets\".\n1. **Compute the terrain gradient.** This is used to determine where water and sediment will flow, as well as the velocity of water at each point.\n1. **Determine the sediment capacity** for each point. This is affected by the terrain slope, water velocity, and water volume. \n1. **Erode or deposit sediment**. If the sediment level is above the capacity, then sediment is deposited to terrain. Otherwise, terrain is eroded into sediment.\n1. **Displace water and sediment downhill.** \n1. **Evaporate** some fraction of the water away.\n\n\nApply this process for long enough and you may get something like this:\n\n

\n \n \n
Terrain from simulated erosion. See here for a time lapse.\n

\n\nThe results are fairly convincing. The tendril-like shape of ridges and cuts you see in real-life terrain are readily apparent. What also jumps out are the large, flat valleys that are the result of sediment deposition over time. If this simulation were left to continue indefinitely, eventually all mountains would be eroded into these flat sedimentary valleys.\n\nBecause of results like you see above, this method of generating terrain can be seen in professional terrain-authoring tools. The code for the terrain above is largely a vectorized implementation of the code found on [this page](http://ranmantaru.com/blog/2011/10/08/water-erosion-on-heightmap-terrain/). For a more theoretical approach, check out this [paper](https://hal.inria.fr/inria-00402079/document).\n\n\n### Pros\n\n* Lot of real-life terrain features simply emerge from running these rules, including stream downcutting, smooth valleys, and differential erosion.\n* Instead of using global parameter values, different regions can be parameterized differently to develop distinct terrain features (e.g. deserts can evolve differently than forests).\n* Fairly easy to parallelize given how straightforward vectorization is.\n\n### Cons\n\n* Parameter hell. There are around 10 constants that need to be set, in addition to other factors like the precipitation pattern and the initial terrain shape. Small changes to any of these can produce completely different results, so it can be difficult to find the ideal combination of parameters that produces good results.\n* Fairly inefficient. Given an NxN grid, in order for changes on one side of the map to affect the opposite size you need O(N) iterations, which puts the overall runtime at O(N3). This means that doubling the grid dimension can result in 8x execution time. This performance cost further exacerbates the cost of parameter tweaking.\n* Difficult to utilize to produce novel terrain. The results of simulation all look like reasonable approximations of real life terrain, however extending this to new types of terrain requires an understanding of the physical processes that would give way to that terrain, which can be prohibitively difficult. \n\n\n## Machine Learning\n\nMachine learning is frequently uses as a panacea for all sorts of problems, and terrain generation is no exception. Machine learning can be effective so long as you have lots of compute power and a large, diverse dataset. Fortunately, compute power is easy to acquire, and lots of terrain elevation data is readily available to download.\n \nThe most suitable machine learning approach is to use a **Generative Adversarial Network (GAN)**. GANs are able to produce fairly convincing novel instances of a distribution described by training data. It works via two neural networks: one that produces new instances of the distribution (called the \"generator\"), and another whose job is to determine whether a provided terrain sample is real (i.e. from the training set), or fake (i.e. via the generator). For some more technical background, check out [these Stanford lectures](https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv).\n\nCreating the right network and tuning all the different hyperparameters can be difficult and requires a lot of expertise to get right. Instead of creating the network from scratch, I will be building off of the work done for *Progressive Growing of GANs for Improved Quality, Stability, and Variation* by Karras, et al. ([paper](https://arxiv.org/pdf/1710.10196.pdf), [code](https://github.com/tkarras/progressive_growing_of_gans)). The basic approach of this paper is to train the network on lower resolution versions of the training samples while adding new layers for progressively higher resolutions. This makes the network converge quicker for high resolution images than it would if training from full resolution images to begin with.\n\n### Training\n\nLike with almost all machine learning projects, most effort is spent in data gathering, cleaning, validation, and training. \n\nThe first step is to get real life terrain height data. For this demonstration, I used the [National Elevation Dataset (NED)](https://lta.cr.usgs.gov/NED) by the USGS. The dataset I used consists of ~1000 1x1 degree height maps with resolutions of 3600x3600 (i.e. pixel size of 1 arcsecond2).\n\nFrom these height maps I will take 512x512 samples for use in training. In the source height arrays, each pixel is a square arcsecond, which means that each sample as-is will appear horizontally stretched, since a square arcsecond is spatially narrower than it is tall. After compensating for this, I also apply several heuristics to filter out what are likely sample unsuitable for training:\n\n* Only accept samples who minimum and maximum elevation span a certain threshold. This approach prefers samples that are more \"mountainous\", and will therefore produce more noticeable erosion effects.\n* Ignore samples if a certain percentage of grid points are within a certain margin of the sample's minimum elevation. This filters out samples that are largely flat, or ones that consist mostly of water.\n* Ignore samples whose [Shannon entropy](https://en.wikipedia.org/wiki/Entropy_(information_theory)) is below a certain threshold. This helps filter out samples that have been corrupted (perhaps due to different libraries used to encode and decode the height data).\n\nIn addition, if we assume that terrain features do not have a directional preference, we can rotate each sample by 90° increments as well as flipping it to increase the dataset size by 8x. In the end, this nets us around 180,000 training samples.\n\nThese training samples are then used to train the GAN. Even using progressively grown GANs, this will still take quite a while to complete (expect around a week even with a beefy Nvidia Tesla GPU).\n\n[Here](https://drive.google.com/file/d/1zdlgpkQu2zqWKJr23di73-lc3hJBAfqW/view?usp=sharing) is a timelapse video several terrain samples throughout the training process.\n\n\n### Results\n\nOnce the network is trained, all we need to do is feed it a new random latent vector into the generator to create new terrain samples:\n\n

\n \n \n
\n \n \n
\n \n \n
\n \n \n
\n \n \n ML-generated terrain.\n

\n\n\n### Pros\n\n* Generated terrain is basically indistinguishable from real-world elevation data. It captures not just erosion effects, but many other natural phenomena that shape terrain in nature.\n* Generation is fairly efficient. Once you have a trained network, creating new terrain samples is fairly fast.\n\n### Cons\n\n* Training is *very* expensive (both in time and money). Lot of effort is required to acquire, clean, validate, and finally train the network. It took about 8 days to train the network used in the above examples.\n* Very little control over the final product. The quality of generated terrain is basically driven by the training samples. Not only do you need a large number of training samples to generate good terrain, you also need good heuristics to make sure that each training sample is suitable. Because training takes so long, it isn't really practical to iterate on these heuristics to generate good results.\n* Difficult to scale to higher resolutions. GANs are generally good a low resolution images. It gets much more expensive, both in terms of compute and memory costs, to scale up to higher resolution height maps.\n\n\n## River Networks\n\nIn most procedural erosion techniques, terrain is carved out first and river placement happens afterward. An alternative method is to work backward: first generate where rivers and streams will be located, and from there determine how the terrain would be shaped to match the rivers. This eases the burden of creating river-friendly terrain by simply defining where the rivers are up front and working the terrain around them.\n\n### Creating the River Network\n\nEvery stream eventually terminates somewhere, most frequently the ocean (they occasionally drain into inland bodies of water, but we will be ignoring those; these drainage basins are called [endorheic basins](https://en.wikipedia.org/wiki/Endorheic_basin)). Given that we need some ocean to drain into, this terrain will be generated as an island,\n\nFirst we start off with what regions will be land or water. Using some simple fBm filtering, we get something like this:\n\n

\n \n
Land mask. Black is ocean, and white is land.\n

\n\nThe next step is to define the nodes on which the river network will be generated. A straightforward approach is to assign a node to each (x, y) coordinate of the image, however this has a tendency to create horizontal and vertical artifacts in the final product. Instead will we create out nodes by sampling some random points across the grid using [Poisson disc sampling](https://www.jasondavies.com/poisson-disc/). After that we use [Delaunay triangulation](https://en.wikipedia.org/wiki/Delaunay_triangulation) to connect the nodes.\n\n

\n \n \n
Left are points via Poisson disc sampling. Right is their Delaunay triangulation.
\n The point spacing in these images is larger than what is used to generate the final terrain.\n \n

\n\nNext, we generate the generic shape the terrain will have (which will later be \"carved out\" via erosion). Because endorheic basins are being avoided in this demo, this terrain is generated such that each point has a downhill path to the ocean (i.e. no landlocked valleys). Here is an example of such a shape:\n\n

\n \n
Initial shape our terrain will take.\n

\n\nThe next step is to generate the river network. The general approach is to generate rivers starting from the mouth (i.e. where they terminate in the ocean) and growing the graph upstream one edge at a time until no more valid edges are left. A valid edge is one that:\n\n* Moves uphill. Since we are growing the river graphs upstream, the end effect is only downhill-flowing rivers.\n* Does not reconnect with an existing river graph. This results in rivers that only merge as they flow downhill, but never split.\n\nFurthermore, we also prioritize which edge to add by how much it aligns with the previous edge in the graph. Without this, rivers will twist and turn in ways that don't appear natural. Furthermore, amount of \"directional inertia\" for each edge can be configured to get more twisty or straight rivers.\n\n

\n \n \n
River networks. Left and right have low and high directional inertia, respectively.\n

\n\nAfter this, the water volume for each node in the river graph is calculated. This is basically done by giving each node a base water volume and adding the sum of all upstream nodes' volumes.\n\n

\n \n
River network with water volume.\n

\n\n\n### Generating the Terrain\n\nThe next step is to generate the terrain height to match the river network. Each node of the river network graph will be assigned a height that will then be rendered via triangulation to get the final height map as a 2D grid.\n\nThe graph traversal move uphill, starting from the water level. Each time an edge is traversed, the height of the next node will be proportional to the height difference in the initial terrain height generated earlier, scaled inversely by the volume of water along that edge. Furthermore, we will cap the height delta between any two nodes to give a thermal-erosion-like effect.\n\nTraversing only the river network edges will produce discontinuities in the generated height, since no two distinct river \"trees\" can communicate with each other. When traversing, we will have to also allow traversing edges that span different river trees. For these edges, we simply assume the edge's water volume to be zero.\n\nIn the end, you get something like this:\n\n

\n \n \n
Final terrain height map from river networks.\n

\n\nIf you're interested in an approach that blends river networks and simulation, check out [this paper](https://hal.inria.fr/hal-01262376/document).\n\n### Pros\n\n* Creates very convincing erosion-like ridges and cuts. The shape of the river network can easily be seen in the generated height map.\n* Easy to add rivers if desired given the already-generated river network.\n* Fairly efficient. Given an NxN height map, this algorithm takes O(N2log N) time.\n\n### Cons\n\n* This algorithm is good at carving out mountains, but needs work to generate other erosion effects like sediment deposition and differential erosion.\n* Some of the algorithms used in this approach are a bit more difficult to parallelize (e.g. best first search).\n\n\n## Running the Code\n\nAll the examples were generated with Python 3.6.0 using Numpy. I've gotten this code to work on OSX and Linux, but I haven't tried with Windows.\n\nMost of the height maps above are generated by running a single python script, with the exception of machine learning which is a bit more involved (described farther down).\n\nHere is a breakdown of all the simple terrain-generating scripts. All outputs are 512x512 grids.\n\n| File | Output | Description\n|:--- | :--- | :---\n| `plain_old_fbm.py` | `fbm.npy` | Regular fBm\n| `domain_warping.py` | `domain_warping.npy` | fBm with domain warping\n| `ridge_noise.py` | `ridge.npy` | The noise with ridge-like effects seen above.\n| `simulation.py` | `simulation.npy` | Eroded terrain via simulation.\n| `river_network.py` | `river_network.npz` | Eroded terrain using river networks. The NPZ file also contains the height map\n\nTo generate the images used in this demo, use the `make_grayscale_image.py` and `make_hillshaded_image.py` scripts. Example: `python3 make_hillshaded_image.py input.npy output.png`\n\n\n### Machine Learning\n\nThe machine learning examples are all heavily dependent on the [Progressive Growing of GANs](https://github.com/tkarras/progressive_growing_of_gans) project, so make sure to clone that repository. That project uses Tensorflow, and requires that you run on a machine with a GPU. If you have a GPU but Tensorflow doesn't see it, you probably have driver issues.\n\n#### Creating the Training Data \n\nIf you wish to train a custom network, you can use whatever source of data you want. For the above examples, I used the USGS.\n\nThe first step is to get the list of URLs pointing to the elevation data:\n\n1. Go to the USGS [download application](https://viewer.nationalmap.gov/basic/)\n1. Select the area from which you want to get elevation data.\n1. On the left under **Data**, select **Elevation Product (3DEP)**, then **1 arc-second DEM**. You can choose other resolutions, but I found 1 arcsecond to be adequate.\n1. Under **File Format**, make sure to select **IMG**.\n1. Click on the **Find Products** button.\n1. Click **Save as CSV**. If you wish to use your own download manager, also click **Save as Text**.\n\nThe next step is to download the actual elevation data. You can either use the `python3 download_ned_zips.py ` which will download the files in the `zip_files/` directory. The USGS site gives this [guide](https://viewer.nationalmap.gov/uget-instructions/) to downloading the files via uGet.\n\nThe next step is to convert the elevation data from IMG files in a ZIP archive to Numpy array files. You can do this by calling `python3 extract_height_arrays.py `. This will write the Numpy arrays to the `array_files/` directory.\n\nAfter this, run `python3 generate_training_images.py`, which will go through each array in the `array_files/` directory, and create 512x512 training sample images from it (written to the `training_samples/` directory). This script performs the validation and filtering described above. It also takes a long time to run, so brew a pot of coffee before you kick it off.\n\nThe next steps will require that you cloned the `progressive_growing_of_gans` project. First, you need to generate the training data in the `tfrecords` format. This can be done by calling:\n\n`progressive_growing_of_gans/: python3 dataset_tool.py /path/to/erosion_3_ways datasets/terrain`\n\nI chose `terrain` as the output directory, but you can use whatever you want (just make sure it's in the `datasets/` directory.\n\nAlmost there! The next step is to edit `config.py` and add the following line to the dataset section:\n\n`desc += '-terrain'; dataset = EasyDict(tfrecord_dir='terrain')`\n\nMake sure to uncomment/delete the \"celebahq\" line. \n\nNow, you can finally run `python3 train.py`. Even with a good graphics card, this will take days to run. For further training customizations, check out [this section](https://github.com/tkarras/progressive_growing_of_gans#preparing-datasets-for-training).\n\nWhen you're done, the `results/` directory will contain all sorts of training outputs, including progress images, Tensorboard logs, and (most importantly) the PKL files containing the network weights.\n\n#### Generating Terrain Samples\n\nTo generate samples, run the following script:\n\n```python3 generate_ml_output.py path/to/progressive_growing_of_gans network_weights.pkl 10```\n\nThe arguments are:\n\n1. The path to the cloned `progressive_growing_of_gans` repository.\n1. The network weights file (the one used for this demo can be found [here](https://drive.google.com/file/d/1czHFcF2ZG_lki7TAQyYCoqtsVcJmdCUN/view?usp=sharing)).\n1. The number of terrain samples to generate (optional, defaults to 20)\n\nThe outputs are written to the `ml_outputs` directory.\n" }, { "path": "domain_warping.py", "content": "#!/usr/bin/python3\n\n# A simple domain warping example.\n\nimport numpy as np\nimport sys\nimport util\n\n\ndef main(argv):\n shape = (512,) * 2\n\n values = util.fbm(shape, -2, lower=2.0)\n offsets = 150 * (util.fbm(shape, -2, lower=1.5) +\n 1j * util.fbm(shape, -2, lower=1.5))\n result = util.sample(values, offsets)\n np.save('domain_warping', result)\n\n\nif __name__ == '__main__':\n main(sys.argv)\n" }, { "path": "download_ned_zips.py", "content": "#!/usr/bin/python3\n\n# Script to hep in downloading files from USGS. The input file is a CSV\n# of 3DEP IMG resources downloaded from https://viewer.nationalmap.gov/basic/\n# This script is really a poor man's download manager; use any solution you \n# prefer.\n\nimport os\nimport shutil\nimport sys\nimport tempfile\nimport time\nimport urllib\nimport util\n\n\n# Gets list of src ids of previously downloaded files.\ndef get_previously_downloaded_ids(dir_path):\n return set((os.path.splitext(file_path)[0]\n for file_path in os.listdir(dir_path)))\n\n\n# Download the file for `src_id` from `url` to `output_dir`. Uses `tmp_dir` an\n# intermediate so that aborted downloads are not included in\n# get_previously_downloaded_ids above.\ndef download_file(src_id, url, output_dir, tmp_dir):\n output_file = src_id + '.zip'\n tmp_path = os.path.join(tmp_dir, output_file)\n\n # Download and save to temp dir\n try:\n urllib.urlretrieve(url, tmp_path)\n except IOError as e:\n print(e)\n return False\n\n # Move file in temp dir to final output dir\n shutil.move(tmp_path, output_dir)\n\n return True\n\n\ndef main(argv):\n my_dir = os.path.dirname(argv[0])\n output_dir = os.path.join(my_dir, 'zip_files')\n tmp_dir = '/tmp'\n\n if len(argv) != 2:\n print('Usage: %s ' % (argv[0]))\n sys.exit(-1)\n\n csv_path = argv[1]\n\n try: os.mkdir(output_dir)\n except: pass\n\n downloaded_ids = get_previously_downloaded_ids(output_dir)\n entries = util.read_csv(csv_path)\n\n for index, entry in enumerate(entries):\n src_id = entry['sourceId']\n download_url = entry['downloadURL']\n pretty_size = entry['prettyFileSize']\n data_format = entry['format']\n\n # Don't download the same file more than once.\n if src_id in downloaded_ids:\n print('Skipping %s' % src_id)\n continue\n\n print('(%d / %d) Processing %s of size %s from %s'\n % (index + 1, len(entries), src_id, pretty_size, download_url))\n\n # Simple data format sanity check.\n if entry['format'] != 'IMG':\n print('Unknown format %s, ignoring...' % (data_format,))\n continue\n\n # Download file.\n if not download_file(src_id, download_url, output_dir, tmp_dir):\n print('Failed to download from %s' % download_url)\n continue\n\n\nif __name__ == '__main__':\n main(sys.argv)\n" }, { "path": "extract_height_arrays.py", "content": "#!/usr/bin/python3\n\n# Extracts the underlying heightmap in each file in zip_files/ and writes the \n# numpy array to array_files/\n\nimport csv\nimport json\nimport numpy as np\nimport os\nfrom osgeo import gdal\nimport re\nimport shutil\nimport sys\nimport tempfile\nimport util\nimport zipfile\n\n\n# Extracts the IMG file from the ZIP archive and returns a Numpy array, or None # if reading or parsing failed.\ndef get_img_array_from_zip(zip_file, img_name):\n with tempfile.NamedTemporaryFile() as temp_file:\n # Copy to temp file.\n with zip_file.open(img_name) as img_file:\n shutil.copyfileobj(img_file, temp_file)\n\n # Extract as numpy array.\n geo = gdal.Open(temp_file.name)\n return geo.ReadAsArray() if geo is not None else None\n\n\ndef main(argv):\n my_dir = os.path.dirname(argv[0])\n input_dir = os.path.join(my_dir, 'zip_files')\n output_dir = os.path.join(my_dir, 'array_files')\n\n if len(argv) != 2:\n print('Usage: %s ' % (argv[0]))\n sys.exit(-1)\n\n csv_path = argv[1]\n\n # Make the output directory if it doesn't exist yet.\n try: os.mkdir(output_dir)\n except: pass\n\n entries = util.read_csv(csv_path)\n for index, entry in enumerate(entries):\n src_id = entry['sourceId']\n print('(%d / %d) Processing %s' % (index + 1, len(entries), src_id))\n zip_path = os.path.join(input_dir, src_id + '.zip')\n\n try:\n # Go though each zip file.\n with zipfile.ZipFile(zip_path, mode='r') as zf:\n ext_names = [name for name in zf.namelist()\n if os.path.splitext(name)[1] == '.img']\n # Check if EXT files.\n if len(ext_names) == 0:\n print('No IMG files found for %s' % (src_id))\n continue;\n\n # Warn if there is more than one IMG file\n if len(ext_names) > 1:\n print('More than one IMG file found for %s: %s' % (src_id, ext_names))\n\n # Get the bounding box. The string manipulation is required given that \n # the provided dict is not proper JSON\n bounding_box_raw = entry['boundingBox']\n bounding_box_json = re.sub(r'([a-zA-Z]+):', r'\"\\1\":', bounding_box_raw)\n bounding_box = json.loads(bounding_box_json)\n\n # Create numpy array from IMG file and write it to output\n array = get_img_array_from_zip(zf, ext_names[0])\n if array is not None:\n output_path = os.path.join(output_dir, src_id + '.npz')\n np.savez(output_path, height=array, **bounding_box)\n else:\n print('Failed to load array for %s' % src_id)\n \n\n except (zipfile.BadZipfile, IOError) as e:\n # Invalid or missing ZIP file.\n print(e)\n continue\n \n\nif __name__ == '__main__':\n main(sys.argv)\n" }, { "path": "generate_ml_output.py", "content": "#!/usrb/bin/python3\n\n# Prouces terrain samples from the trained generator network.\n\nimport numpy as np\nimport os\nimport pickle\nimport sys\nimport tensorflow as tf\n\n\ndef main(argv):\n # First argument is the path to the progressive_growing_of_gans clone. This\n # is needed to for proper loading of the weights via pickle.\n # Second argument is the network weights pickle file.\n # Third argument is the number of output samples to generate. Defaults to 20\n if len(argv) < 3:\n print('Usage: %s path/to/progressive_growing_of_gans weights.pkl '\n '[number_of_samples]' % argv[0])\n sys.exit(-1)\n my_dir = os.path.dirname(argv[0])\n pgog_path = argv[1]\n weight_path = argv[2]\n num_samples = int(argv[3]) if len(argv) >= 4 else 20\n\n # Load the GAN tensors.\n tf.InteractiveSession()\n sys.path.append(pgog_path)\n with open(weight_path, 'rb') as f:\n G, D, Gs = pickle.load(f)\n\n # Generate input vectors.\n latents = np.random.randn(num_samples, *Gs.input_shapes[0][1:])\n labels = np.zeros([latents.shape[0]] + Gs.input_shapes[1][1:])\n\n # Run generator to create samples.\n samples = Gs.run(latents, labels)\n\n # Make output directory\n output_dir = os.path.join(my_dir, 'ml_outputs')\n try: os.mkdir(output_dir)\n except: pass\n\n # Write outputs.\n for idx in range(samples.shape[0]):\n sample = (np.clip(np.squeeze((samples[idx, 0, :, :] + 1.0) / 2), 0.0, 1.0)\n .astype('float64'))\n np.save(os.path.join(output_dir, '%d.npy' % idx), sample)\n\n\nif __name__ == '__main__':\n main(sys.argv)\n" }, { "path": "generate_training_images.py", "content": "#!/usr/bin/python\n\n# Reads the Numpy arrays in array_files/ and generates images for use in\n# training. Please note that this script takes a long time to run.\n\nimport cv2\nimport numpy as np\nimport skimage.measure\nimport os\nimport sys\nimport util\n\n\n# Filters and cleans the given sample. Uses rough heuristics to determine which\n# samples are suitable for training via rough heuristics.\ndef clean_sample(sample):\n # Get rid of \"out-of-bounds\" magic values.\n sample[sample == np.finfo('float32').min] = 0.0\n\n # Ignore any samples with NaNs, for one reason or another.\n if np.isnan(sample).any(): return None\n\n # Only accept values that span a given range. This is to capture more\n # mountainous samples.\n if (sample.max() - sample.min()) < 40: return None\n \n # Filter out samples for which a significant portion is within a small \n # threshold from the minimum value. This helps filter out samples that\n # contain a lot of water.\n near_min_fraction = (sample < (sample.min() + 8)).sum() / sample.size\n if near_min_fraction > 0.2: return None\n\n # Low entropy samples likely have some file corruption or some other artifact\n # that would make it unsuitable as a training sample.\n entropy = skimage.measure.shannon_entropy(sample)\n if entropy < 10.0: return None\n\n return util.normalize(sample)\n\n\n# This function returns rotated and flipped variants of the provided array. This\n# increases the number of training samples by a factor of 8.\ndef get_variants(a):\n for b in (a, a.T): # Original and flipped.\n for k in range(0, 4): # Rotated 90 degrees x 4\n yield np.rot90(b, k)\n\n\ndef main(argv):\n my_dir = os.path.dirname(argv[0])\n source_array_dir = os.path.join(my_dir, 'array_files')\n training_samples_dir = os.path.join(my_dir, 'training_samples')\n sample_dim = 512\n sample_shape = (sample_dim,) * 2\n sample_area = np.prod(sample_shape)\n\n # Create the training sample directory, if it doesn't already exist.\n try: os.mkdir(training_samples_dir)\n except: pass\n \n source_array_paths = [os.path.join(source_array_dir, path)\n for path in os.listdir(source_array_dir)]\n\n training_id = 0\n for (index, source_array_path) in enumerate(source_array_paths):\n print('(%d / %d) Created %d samples so far'\n % (index + 1, len(source_array_paths), training_id))\n data = np.load(source_array_path)\n\n # Load heightmap and correct for latitude (to an approximation)\n source_array_raw = data['height']\n latitude_deg = (data['minY'] + data['maxY']) / 2\n latitude_correction = np.cos(np.radians(latitude_deg))\n source_array_shape = (\n int(np.round(source_array_raw.shape[0] * latitude_correction)),\n source_array_raw.shape[1])\n source_array = cv2.resize(source_array_raw, source_array_shape)\n\n # Determine the number of samples to use per source array.\n sampleable_area = np.subtract(source_array_shape, sample_shape).prod()\n samples_per_array = int(np.ceil(sampleable_area / sample_area))\n\n if len(source_array.shape) == 0:\n print('Invalid array at %s' % source_array_path)\n continue\n\n for _ in range(samples_per_array):\n # Select a sample from the source array.\n row = np.random.randint(source_array.shape[0] - sample_shape[0])\n col = np.random.randint(source_array.shape[1] - sample_shape[1])\n sample = source_array[row:(row + sample_shape[0]),\n col:(col + sample_shape[1])]\n\n # Scale and clean the sample\n sample = clean_sample(sample)\n\n # Write the sample to a file\n if sample is not None:\n for variant in get_variants(sample):\n output_path = os.path.join(\n training_samples_dir, str(training_id) + '.png')\n util.save_as_png(variant, output_path)\n\n training_id += 1\n\n\nif __name__ == '__main__':\n main(sys.argv)\n" }, { "path": "make_grayscale_image.py", "content": "#!/usr/bin/python3\n\n# Genreates a PNG containing the terrain height in grayscale.\n\nimport util\nimport sys\n\n\ndef main(argv):\n if len(argv) != 3:\n print('Usage: %s ' % (argv[0],))\n sys.exit(-1)\n\n input_path = argv[1]\n output_path = argv[2]\n\n height, _ = util.load_from_file(input_path)\n util.save_as_png(height, output_path)\n\n\nif __name__ == '__main__':\n main(sys.argv)\n" }, { "path": "make_hillshaded_image.py", "content": "#!/usr/bin/python3\n\n# Genreates a PNG containing a hillshaded version of the terrain height.\n\nimport util\nimport sys\n\n\ndef main(argv):\n if len(argv) != 3:\n print('Usage: %s ' % (argv[0],))\n sys.exit(-1)\n\n input_path = argv[1]\n output_path = argv[2]\n\n height, land_mask = util.load_from_file(input_path)\n util.save_as_png(util.hillshaded(height, land_mask=land_mask), output_path)\n\n\nif __name__ == '__main__':\n main(sys.argv)\n" }, { "path": "plain_old_fbm.py", "content": "#!/usr/bin/python3\n\n# A demo of just regular FBM noise\n\nimport numpy as np\nimport sys\nimport util\n\n\ndef main(argv):\n shape = (512,) * 2\n np.save('fbm', util.fbm(shape, -2, lower=2.0))\n\n\nif __name__ == '__main__':\n main(sys.argv)\n" }, { "path": "requirements-pip3.txt", "content": "# GDAL is a bit janky, so you may have to compile and install yourself.\n# See https://trac.osgeo.org/gdal/wiki/DownloadSource\nGDAL==2.3.2\nmatplotlib==3.0.0\nnumpy==1.15.2\nopencv-python==3.4.3.18\nPillow==8.2.0\nscikit-image==0.14.1\nscipy==1.1.0\nsix==1.11.0\n# You will also have to install CUDA and cuDNN drivers for tensorflow to work.\ntensorboard==1.11.0\ntensorflow==2.5.0\ntensorflow-gpu==2.3.1\ntensorflow-tensorboard==1.5.1\nurllib3==1.26.5\n" }, { "path": "ridge_noise.py", "content": "#!/usr/bin/python3\n\n# A demo of ridge noise.\n\nimport numpy as np\nimport sys\nimport util\n\ndef noise_octave(shape, f):\n return util.fbm(shape, -1, lower=f, upper=(2 * f))\n\ndef main(argv):\n shape = (512,) * 2\n\n values = np.zeros(shape)\n for p in range(1, 10):\n a = 2 ** p\n values += np.abs(noise_octave(shape, a) - 0.5)/ a \n result = (1.0 - util.normalize(values)) ** 2\n\n np.save('ridge', result)\n\n\nif __name__ == '__main__':\n main(sys.argv)\n" }, { "path": "river_network.py", "content": "#!/usr/bin/python3\n\nimport collections\nimport heapq\nimport numpy as np\nimport matplotlib\nimport matplotlib.collections as mc\nimport matplotlib.pyplot as plt\nimport scipy as sp\nimport scipy.spatial\nimport skimage.measure\nimport sys\nimport util\n\n\n# Returns the index of the smallest value of `a`\ndef min_index(a): return a.index(min(a))\n\n\n# Returns an array with a bump centered in the middle of `shape`. `sigma`\n# determines how wide the bump is.\ndef bump(shape, sigma):\n [y, x] = np.meshgrid(*map(np.arange, shape))\n r = np.hypot(x - shape[0] / 2, y - shape[1] / 2)\n c = min(shape) / 2\n return np.tanh(np.maximum(c - r, 0.0) / sigma)\n\n\n# Returns a list of heights for each point in `points`.\ndef compute_height(points, neighbors, deltas, get_delta_fn=None):\n if get_delta_fn is None:\n get_delta_fn = lambda src, dst: deltas[dst]\n\n dim = len(points)\n result = [None] * dim\n seed_idx = min_index([sum(p) for p in points])\n q = [(0.0, seed_idx)]\n\n while len(q) > 0:\n (height, idx) = heapq.heappop(q)\n if result[idx] is not None: continue\n result[idx] = height\n for n in neighbors[idx]:\n if result[n] is not None: continue\n heapq.heappush(q, (get_delta_fn(idx, n) + height, n))\n return util.normalize(np.array(result))\n\n\n# Same as above, but computes height taking into account river downcutting.\n# `max_delta` determines the maximum difference in neighboring points (to\n# give the effect of talus slippage). `river_downcutting_constant` affects how\n# deeply rivers cut into terrain (higher means more downcutting).\ndef compute_final_height(points, neighbors, deltas, volume, upstream,\n max_delta, river_downcutting_constant):\n dim = len(points)\n result = [None] * dim\n seed_idx = min_index([sum(p) for p in points])\n q = [(0.0, seed_idx)]\n\n def get_delta(src, dst):\n v = volume[dst] if (dst in upstream[src]) else 0.0\n downcut = 1.0 / (1.0 + v ** river_downcutting_constant) \n return min(max_delta, deltas[dst] * downcut)\n\n return compute_height(points, neighbors, deltas, get_delta_fn=get_delta)\n\n\n# Computes the river network that traverses the terrain.\n# Arguments:\n# * points: The (x,y) coordinates of each point\n# * neghbors: Set of each neighbor index for each point.\n# * heights: The height of each point.\n# * land: Indicates whether each point is on land or water.\n# * directional_interta: indicates how straight the rivers are\n# (0 = no directional inertia, 1 = total directional inertia).\n# * default_water_level: How much water is assigned by default to each point\n# * evaporation_rate: How much water is evaporated as it traverses from along\n# each river edge.\n# \n# Returns a 3-tuple of:\n# * List of indices of all points upstream from each point \n# * List containing the index of the point downstream of each point.\n# * The water volume of each point.\ndef compute_river_network(points, neighbors, heights, land,\n directional_inertia, default_water_level,\n evaporation_rate):\n num_points = len(points)\n\n # The normalized vector between points i and j\n def unit_delta(i, j):\n delta = points[j] - points[i]\n return delta / np.linalg.norm(delta)\n\n # Initialize river priority queue with all edges between non-land points to\n # land points. Each entry is a tuple of (priority, (i, j, river direction))\n q = []\n roots = set()\n for i in range(num_points):\n if land[i]: continue\n is_root = True\n for j in neighbors[i]:\n if not land[j]: continue\n is_root = True\n heapq.heappush(q, (-1.0, (i, j, unit_delta(i, j))))\n if is_root: roots.add(i)\n\n # Compute the map of each node to its downstream node.\n downstream = [None] * num_points\n\n while len(q) > 0:\n (_, (i, j, direction)) = heapq.heappop(q)\n\n # Assign i as being downstream of j, assuming such a point doesn't\n # already exist.\n if downstream[j] is not None: continue\n downstream[j] = i\n\n # Go through each neighbor of upstream point j.\n for k in neighbors[j]:\n # Ignore neighbors that are lower than the current point, or who already \n # have an assigned downstream point.\n if (heights[k] < heights[j] or downstream[k] is not None\n or not land[k]):\n continue\n\n # Edges that are aligned with the current direction vector are\n # prioritized.\n neighbor_direction = unit_delta(j, k)\n priority = -np.dot(direction, neighbor_direction)\n\n # Add new edge to queue.\n weighted_direction = util.lerp(neighbor_direction, direction,\n directional_inertia)\n heapq.heappush(q, (priority, (j, k, weighted_direction)))\n\n\n # Compute the mapping of each node to its upstream nodes.\n upstream = [set() for _ in range(num_points)]\n for i, j in enumerate(downstream):\n if j is not None: upstream[j].add(i)\n\n # Compute the water volume for each node.\n volume = [None] * num_points\n def compute_volume(i):\n if volume[i] is not None: return\n v = default_water_level\n for j in upstream[i]:\n compute_volume(j)\n v += volume[j]\n volume[i] = v * (1 - evaporation_rate)\n\n for i in range(0, num_points): compute_volume(i)\n\n return (upstream, downstream, volume)\n\n\n# Renders `values` for each triangle in `tri` on an array the size of `shape`.\ndef render_triangulation(shape, tri, values):\n points = util.make_grid_points(shape)\n triangulation = matplotlib.tri.Triangulation(\n tri.points[:,0], tri.points[:,1], tri.simplices)\n interp = matplotlib.tri.LinearTriInterpolator(triangulation, values)\n return interp(points[:,0], points[:,1]).reshape(shape).filled(0.0)\n\n\n# Removes any bodies of water completely enclosed by land.\ndef remove_lakes(mask):\n labels = skimage.measure.label(mask)\n new_mask = np.zeros_like(mask, dtype=bool)\n labels = skimage.measure.label(~mask, connectivity=1)\n new_mask[labels != labels[0, 0]] = True\n return new_mask\n\n\ndef main(argv):\n dim = 512\n shape = (dim,) * 2\n disc_radius = 1.0\n max_delta = 0.05\n river_downcutting_constant = 1.3\n directional_inertia = 0.4\n default_water_level = 1.0\n evaporation_rate = 0.2\n\n print ('Generating...')\n\n print(' ...initial terrain shape')\n land_mask = remove_lakes(\n (util.fbm(shape, -2, lower=2.0) + bump(shape, 0.2 * dim) - 1.1) > 0)\n coastal_dropoff = np.tanh(util.dist_to_mask(land_mask) / 80.0) * land_mask\n mountain_shapes = util.fbm(shape, -2, lower=2.0, upper=np.inf)\n initial_height = ( \n (util.gaussian_blur(np.maximum(mountain_shapes - 0.40, 0.0), sigma=5.0) \n + 0.1) * coastal_dropoff)\n deltas = util.normalize(np.abs(util.gaussian_gradient(initial_height))) \n\n print(' ...sampling points')\n points = util.poisson_disc_sampling(shape, disc_radius)\n coords = np.floor(points).astype(int)\n\n\n print(' ...delaunay triangulation')\n tri = sp.spatial.Delaunay(points)\n (indices, indptr) = tri.vertex_neighbor_vertices\n neighbors = [indptr[indices[k]:indices[k + 1]] for k in range(len(points))]\n points_land = land_mask[coords[:, 0], coords[:, 1]]\n points_deltas = deltas[coords[:, 0], coords[:, 1]]\n\n print(' ...initial height map')\n points_height = compute_height(points, neighbors, points_deltas)\n\n print(' ...river network')\n (upstream, downstream, volume) = compute_river_network(\n points, neighbors, points_height, points_land,\n directional_inertia, default_water_level, evaporation_rate)\n\n print(' ...final terrain height')\n new_height = compute_final_height(\n points, neighbors, points_deltas, volume, upstream, \n max_delta, river_downcutting_constant)\n terrain_height = render_triangulation(shape, tri, new_height)\n\n np.savez('river_network', height=terrain_height, land_mask=land_mask)\n\n\nif __name__ == '__main__':\n main(sys.argv)\n" }, { "path": "simulation.py", "content": "#!/usr/bin/python3\n\n# Semi-phisically-based hydraulic erosion simulation. Code is inspired by the \n# code found here:\n# http://ranmantaru.com/blog/2011/10/08/water-erosion-on-heightmap-terrain/\n# With some theoretical inspiration from here:\n# https://hal.inria.fr/inria-00402079/document\n\nimport numpy as np\nimport scipy as sp\nimport matplotlib.pyplot as plt\nimport os\nimport sys\nimport util\n\n\n# Smooths out slopes of `terrain` that are too steep. Rough approximation of the\n# phenomenon described here: https://en.wikipedia.org/wiki/Angle_of_repose\ndef apply_slippage(terrain, repose_slope, cell_width):\n delta = util.simple_gradient(terrain) / cell_width\n smoothed = util.gaussian_blur(terrain, sigma=1.5)\n should_smooth = np.abs(delta) > repose_slope\n result = np.select([np.abs(delta) > repose_slope], [smoothed], terrain)\n return result\n\n\ndef main(argv):\n # Grid dimension constants\n full_width = 200\n dim = 512\n shape = [dim] * 2\n cell_width = full_width / dim\n cell_area = cell_width ** 2\n\n # Snapshotting parameters. Only needed for generating the simulation\n # timelapse.\n enable_snapshotting = False\n my_dir = os.path.dirname(argv[0])\n snapshot_dir = os.path.join(my_dir, 'sim_snaps')\n snapshot_file_template = 'sim-%05d.png'\n if enable_snapshotting:\n try: os.mkdir(snapshot_dir)\n except: pass\n\n # Water-related constants\n rain_rate = 0.0008 * cell_area\n evaporation_rate = 0.0005\n\n # Slope constants\n min_height_delta = 0.05\n repose_slope = 0.03\n gravity = 30.0\n gradient_sigma = 0.5\n\n # Sediment constants\n sediment_capacity_constant = 50.0\n dissolving_rate = 0.25\n deposition_rate = 0.001\n\n # The numer of iterations is proportional to the grid dimension. This is to \n # allow changes on one side of the grid to affect the other side.\n iterations = int(1.4 * dim)\n\n # `terrain` represents the actual terrain height we're interested in\n terrain = util.fbm(shape, -2.0)\n\n # `sediment` is the amount of suspended \"dirt\" in the water. Terrain will be\n # transfered to/from sediment depending on a number of different factors.\n sediment = np.zeros_like(terrain)\n\n # The amount of water. Responsible for carrying sediment.\n water = np.zeros_like(terrain)\n\n # The water velocity.\n velocity = np.zeros_like(terrain)\n\n for i in range(0, iterations):\n print('%d / %d' % (i + 1, iterations))\n\n # Add precipitation. This is done by via simple uniform random distribution,\n # although other models use a raindrop model\n water += np.random.rand(*shape) * rain_rate\n\n # Compute the normalized gradient of the terrain height to determine where \n # water and sediment will be moving.\n gradient = np.zeros_like(terrain, dtype='complex')\n gradient = util.simple_gradient(terrain)\n gradient = np.select([np.abs(gradient) < 1e-10],\n [np.exp(2j * np.pi * np.random.rand(*shape))],\n gradient)\n gradient /= np.abs(gradient)\n\n # Compute the difference between teh current height the height offset by\n # `gradient`.\n neighbor_height = util.sample(terrain, -gradient)\n height_delta = terrain - neighbor_height\n \n # The sediment capacity represents how much sediment can be suspended in\n # water. If the sediment exceeds the quantity, then it is deposited,\n # otherwise terrain is eroded.\n sediment_capacity = (\n (np.maximum(height_delta, min_height_delta) / cell_width) * velocity *\n water * sediment_capacity_constant)\n deposited_sediment = np.select(\n [\n height_delta < 0, \n sediment > sediment_capacity,\n ], [\n np.minimum(height_delta, sediment),\n deposition_rate * (sediment - sediment_capacity),\n ],\n # If sediment <= sediment_capacity\n dissolving_rate * (sediment - sediment_capacity))\n\n # Don't erode more sediment than the current terrain height.\n deposited_sediment = np.maximum(-height_delta, deposited_sediment)\n\n # Update terrain and sediment quantities.\n sediment -= deposited_sediment\n terrain += deposited_sediment\n sediment = util.displace(sediment, gradient)\n water = util.displace(water, gradient)\n\n # Smooth out steep slopes.\n terrain = apply_slippage(terrain, repose_slope, cell_width)\n\n # Update velocity\n velocity = gravity * height_delta / cell_width\n \n # Apply evaporation\n water *= 1 - evaporation_rate\n\n # Snapshot, if applicable.\n if enable_snapshotting:\n output_path = os.path.join(snapshot_dir, snapshot_file_template % i)\n util.save_as_png(terrain, output_path)\n\n\n np.save('simulation', util.normalize(terrain))\n\n \nif __name__ == '__main__':\n main(sys.argv)\n" }, { "path": "util.py", "content": "# Various common functions.\n\nfrom PIL import Image\nimport collections\nimport csv\nfrom matplotlib.colors import LightSource, LinearSegmentedColormap\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport scipy as sp\nimport scipy.spatial\n\n\n# Open CSV file as a dict.\ndef read_csv(csv_path):\n with open(csv_path, 'r') as csv_file:\n return list(csv.DictReader(csv_file))\n\n\n# Renormalizes the values of `x` to `bounds`\ndef normalize(x, bounds=(0, 1)):\n return np.interp(x, (x.min(), x.max()), bounds)\n\n\n# Fourier-based power law noise with frequency bounds.\ndef fbm(shape, p, lower=-np.inf, upper=np.inf):\n freqs = tuple(np.fft.fftfreq(n, d=1.0 / n) for n in shape)\n freq_radial = np.hypot(*np.meshgrid(*freqs))\n envelope = (np.power(freq_radial, p, where=freq_radial!=0) *\n (freq_radial > lower) * (freq_radial < upper))\n envelope[0][0] = 0.0\n phase_noise = np.exp(2j * np.pi * np.random.rand(*shape))\n return normalize(np.real(np.fft.ifft2(np.fft.fft2(phase_noise) * envelope)))\n\n\n# Returns each value of `a` with coordinates offset by `offset` (via complex \n# values). The values at the new coordiantes are the linear interpolation of\n# neighboring values in `a`.\ndef sample(a, offset):\n shape = np.array(a.shape)\n delta = np.array((offset.real, offset.imag))\n coords = np.array(np.meshgrid(*map(range, shape))) - delta\n\n lower_coords = np.floor(coords).astype(int)\n upper_coords = lower_coords + 1\n coord_offsets = coords - lower_coords \n lower_coords %= shape[:, np.newaxis, np.newaxis]\n upper_coords %= shape[:, np.newaxis, np.newaxis]\n\n result = lerp(lerp(a[lower_coords[1], lower_coords[0]],\n a[lower_coords[1], upper_coords[0]],\n coord_offsets[0]),\n lerp(a[upper_coords[1], lower_coords[0]],\n a[upper_coords[1], upper_coords[0]],\n coord_offsets[0]),\n coord_offsets[1])\n return result\n\n\n# Takes each value of `a` and offsets them by `delta`. Treats each grid point\n# like a unit square.\ndef displace(a, delta):\n fns = {\n -1: lambda x: -x,\n 0: lambda x: 1 - np.abs(x),\n 1: lambda x: x,\n }\n result = np.zeros_like(a)\n for dx in range(-1, 2):\n wx = np.maximum(fns[dx](delta.real), 0.0)\n for dy in range(-1, 2):\n wy = np.maximum(fns[dy](delta.imag), 0.0)\n result += np.roll(np.roll(wx * wy * a, dy, axis=0), dx, axis=1)\n\n return result\n\n\n# Returns the gradient of the gaussian blur of `a` encoded as a complex number. \ndef gaussian_gradient(a, sigma=1.0):\n [fy, fx] = np.meshgrid(*(np.fft.fftfreq(n, 1.0 / n) for n in a.shape))\n sigma2 = sigma**2\n g = lambda x: ((2 * np.pi * sigma2) ** -0.5) * np.exp(-0.5 * (x / sigma)**2)\n dg = lambda x: g(x) * (x / sigma2)\n\n fa = np.fft.fft2(a)\n dy = np.fft.ifft2(np.fft.fft2(dg(fy) * g(fx)) * fa).real\n dx = np.fft.ifft2(np.fft.fft2(g(fy) * dg(fx)) * fa).real\n return 1j * dx + dy\n\n\n# Simple gradient by taking the diff of each cell's horizontal and vertical\n# neighbors.\ndef simple_gradient(a):\n dx = 0.5 * (np.roll(a, 1, axis=0) - np.roll(a, -1, axis=0))\n dy = 0.5 * (np.roll(a, 1, axis=1) - np.roll(a, -1, axis=1))\n return 1j * dx + dy\n\n\n# Loads the terrain height array (and optionally the land mask from the given \n# file.\ndef load_from_file(path):\n result = np.load(path)\n if type(result) == np.lib.npyio.NpzFile:\n return (result['height'], result['land_mask'])\n else:\n return (result, None)\n\n\n# Saves the array as a PNG image. Assumes all input values are [0, 1]\ndef save_as_png(a, path):\n image = Image.fromarray(np.round(a * 255).astype('uint8'))\n image.save(path)\n\n\n# Creates a hillshaded RGB array of heightmap `a`.\n_TERRAIN_CMAP = LinearSegmentedColormap.from_list('my_terrain', [\n (0.00, (0.15, 0.3, 0.15)),\n (0.25, (0.3, 0.45, 0.3)),\n (0.50, (0.5, 0.5, 0.35)),\n (0.80, (0.4, 0.36, 0.33)),\n (1.00, (1.0, 1.0, 1.0)),\n])\ndef hillshaded(a, land_mask=None, angle=270):\n if land_mask is None: land_mask = np.ones_like(a)\n ls = LightSource(azdeg=angle, altdeg=30)\n land = ls.shade(a, cmap=_TERRAIN_CMAP, vert_exag=10.0,\n blend_mode='overlay')[:, :, :3]\n water = np.tile((0.25, 0.35, 0.55), a.shape + (1,))\n return lerp(water, land, land_mask[:, :, np.newaxis])\n\n\n# Linear interpolation of `x` to `y` with respect to `a`\ndef lerp(x, y, a): return (1.0 - a) * x + a * y\n\n\n# Returns a list of grid coordinates for every (x, y) position bounded by\n# `shape`\ndef make_grid_points(shape):\n [Y, X] = np.meshgrid(np.arange(shape[0]), np.arange(shape[1])) \n grid_points = np.column_stack([X.flatten(), Y.flatten()])\n return grid_points\n\n\n# Returns a list of points sampled within the bounds of `shape` and with a\n# minimum spacing of `radius`.\n# NOTE: This function is fairly slow, given that it is implemented with almost\n# no array operations.\ndef poisson_disc_sampling(shape, radius, retries=16):\n grid = {}\n points = []\n\n # The bounds of `shape` are divided into a grid of cells, each of which can\n # contain a maximum of one point.\n cell_size = radius / np.sqrt(2)\n cells = np.ceil(np.divide(shape, cell_size)).astype(int)\n offsets = [(0, 0), (0, -1), (0, 1), (-1, 0), (1, 0), (-1, -1), (-1, 1),\n (1, -1), (1, 1), (-2, 0), (2, 0), (0, -2), (0, 2)]\n to_cell = lambda p: (p / cell_size).astype('int')\n\n # Returns true if there is a point within `radius` of `p`.\n def has_neighbors_in_radius(p):\n cell = to_cell(p)\n for offset in offsets:\n cell_neighbor = (cell[0] + offset[0], cell[1] + offset[1])\n if cell_neighbor in grid:\n p2 = grid[cell_neighbor]\n diff = np.subtract(p2, p)\n if np.dot(diff, diff) <= radius * radius:\n return True\n return False \n\n # Adds point `p` to the cell grid.\n def add_point(p):\n grid[tuple(to_cell(p))] = p\n q.append(p)\n points.append(p)\n\n q = collections.deque()\n first = shape * np.random.rand(2)\n add_point(first)\n while len(q) > 0:\n point = q.pop()\n\n # Make `retries` attemps to find a point within [radius, 2 * radius] from\n # `point`.\n for _ in range(retries):\n diff = 2 * radius * (2 * np.random.rand(2) - 1)\n r2 = np.dot(diff, diff)\n new_point = diff + point\n if (new_point[0] >= 0 and new_point[0] < shape[0] and\n new_point[1] >= 0 and new_point[1] < shape[1] and \n not has_neighbors_in_radius(new_point) and\n r2 > radius * radius and r2 < 4 * radius * radius):\n add_point(new_point)\n num_points = len(points)\n\n # Return points list as a numpy array.\n return np.concatenate(points).reshape((num_points, 2))\n\n\n# Returns an array in which all True values of `mask` contain the distance to\n# the nearest False value.\ndef dist_to_mask(mask):\n border_mask = (np.maximum.reduce([\n np.roll(mask, 1, axis=0), np.roll(mask, -1, axis=0),\n np.roll(mask, -1, axis=1), np.roll(mask, 1, axis=1)]) * (1 - mask))\n border_points = np.column_stack(np.where(border_mask > 0))\n\n kdtree = sp.spatial.cKDTree(border_points)\n grid_points = make_grid_points(mask.shape)\n\n return kdtree.query(grid_points)[0].reshape(mask.shape)\n\n\n# Generates worley noise with points separated by `spacing`.\ndef worley(shape, spacing):\n points = poisson_disc_sampling(shape, spacing)\n coords = np.floor(points).astype(int)\n mask = np.zeros(shape, dtype=bool)\n mask[coords[:, 0], coords[:, 1]] = True\n return normalize(dist_to_mask(mask))\n\n\n# Peforms a gaussian blur of `a`.\ndef gaussian_blur(a, sigma=1.0):\n freqs = tuple(np.fft.fftfreq(n, d=1.0 / n) for n in a.shape)\n freq_radial = np.hypot(*np.meshgrid(*freqs))\n sigma2 = sigma**2\n g = lambda x: ((2 * np.pi * sigma2) ** -0.5) * np.exp(-0.5 * (x / sigma)**2)\n kernel = g(freq_radial)\n kernel /= kernel.sum()\n return np.fft.ifft2(np.fft.fft2(a) * np.fft.fft2(kernel)).real\n" } ]