Repository: zhengjie9510/google-map-downloader
Branch: master
Commit: 549bc133604d
Files: 5
Total size: 28.6 KB
Directory structure:
gitextract_hk_bi296/
├── LICENSE
├── README.md
├── downloader_1.1.py
├── downloader_1.2.py
└── requirements.txt
================================================
FILE CONTENTS
================================================
================================================
FILE: LICENSE
================================================
MIT License
Copyright (c) 2020 zhengjie9510
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
# Google-Map-Downloader

## 中文:
[downloader_1.1](https://github.com/zhengjie9510/Google-Map-Downloader/blob/master/downloader_1.1.py):
一个小工具,你只需要输入空间范围、地图缩放等级就可以实现Google地图的下载,并输出为TIFF格式,含空间坐标系。
[downloader_1.2](https://github.com/zhengjie9510/Google-Map-Downloader/blob/master/downloader_1.2.py):
是在1.1版本上的改进。由于python的多线程中存在GIL锁,导致python的多线程不能利用多核,考虑到现在的计算机是多核的,为了充分利用计算机的多核资源,提高下载速度,尝试利用多进程+多线程的方式来实现地图切片下载,最终速度得到极大提高。但该部分还没有实现进度条功能。
## English:
[downloader_1.1](https://github.com/zhengjie9510/Google-Map-Downloader/blob/master/downloader_1.1.py):
A small tool, you only need to input the spatial extent and map zoom level to download Google Maps, and output to TIFF format, including the spatial coordinate system.
[downloader_1.2](https://github.com/zhengjie9510/Google-Map-Downloader/blob/master/downloader_1.2.py):
It is an improvement on version 1.1. Due to the existence of GIL locks in python's multi-threading, python's multi-threading cannot use multi-cores. Considering that computers are now multi-core, In order to make full use of the computer's multi-core resources and increase the download speed, try to use multi-process + multi-threaded way to achieve map tile download. The final speed has been greatly improved, but this part has not implemented the progress bar function.
## 指南/Guide
### 安装/Install
```python
conda install --yes --file requirements.txt
```
### 使用/Use
```python
if __name__ == '__main__':
start_time=time.time()
# main(100.361,38.866,100.386,38.839,13,r'C:\Users\test.tif')
main(left,top,right,bottom,zoom,filePath,style='s',server="Google")
end_time=time.time()
print('lasted a total of {:.2f} seconds'.format(end_time-start_time))
```
```python
'''
Parameters
----------
left, top : left-top coordinate, for example (100.361,38.866)
right, bottom : right-bottom coordinate
z : zoom
filePath : File path for storing results, TIFF format
style :
m for map;
s for satellite;
y for satellite with label;
t for terrain;
p for terrain with label;
h for label;
source : Google China (default) or Google
'''
```
## 问题/Issues
If you encounter the problem of Bad network link, you can change the HEADERS in the download function, and try again.
```python
def download(self,url):
HEADERS = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.76 Safari/537.36'}
header = ur.Request(url,headers=HEADERS)
err=0
while(err<3):
try:
data = ur.urlopen(header).read()
except:
err+=1
else:
return data
raise Exception("Bad network link.")
```
================================================
FILE: downloader_1.1.py
================================================
# -*- coding: utf-8 -*-
'''
This code is used to download image from google
@date : 2020-3-10
@author: Zheng Jie
@E-mail: zhengjie9510@qq.com
'''
import io
import math
import numpy as np
from math import floor, pi, log, tan, atan, exp
from threading import Thread, Lock
import urllib.request as ur
import PIL.Image as pil
import cv2
from osgeo import gdal, osr
import time
# ------------------Interchange between WGS-84 and Web Mercator-------------------------
# WGS-84 to Web Mercator
def wgs_to_mercator(x, y):
y = 85.0511287798 if y > 85.0511287798 else y
y = -85.0511287798 if y < -85.0511287798 else y
x2 = x * 20037508.34 / 180
y2 = log(tan((90 + y) * pi / 360)) / (pi / 180)
y2 = y2 * 20037508.34 / 180
return x2, y2
# Web Mercator to WGS-84
def mercator_to_wgs(x, y):
x2 = x / 20037508.34 * 180
y2 = y / 20037508.34 * 180
y2 = 180 / pi * (2 * atan(exp(y2 * pi / 180)) - pi / 2)
return x2, y2
# -------------------------------------------------------------
# -----------------Interchange between GCJ-02 to WGS-84---------------------------
# All public geographic data in mainland China need to be encrypted with GCJ-02, introducing random bias
# This part of the code is used to remove the bias
def transformLat(x, y):
ret = -100.0 + 2.0 * x + 3.0 * y + 0.2 * y * y + 0.1 * x * y + 0.2 * math.sqrt(abs(x))
ret += (20.0 * math.sin(6.0 * x * math.pi) + 20.0 * math.sin(2.0 * x * math.pi)) * 2.0 / 3.0
ret += (20.0 * math.sin(y * math.pi) + 40.0 * math.sin(y / 3.0 * math.pi)) * 2.0 / 3.0
ret += (160.0 * math.sin(y / 12.0 * math.pi) + 320 * math.sin(y * math.pi / 30.0)) * 2.0 / 3.0
return ret
def transformLon(x, y):
ret = 300.0 + x + 2.0 * y + 0.1 * x * x + 0.1 * x * y + 0.1 * math.sqrt(abs(x))
ret += (20.0 * math.sin(6.0 * x * math.pi) + 20.0 * math.sin(2.0 * x * math.pi)) * 2.0 / 3.0
ret += (20.0 * math.sin(x * math.pi) + 40.0 * math.sin(x / 3.0 * math.pi)) * 2.0 / 3.0
ret += (150.0 * math.sin(x / 12.0 * math.pi) + 300.0 * math.sin(x / 30.0 * math.pi)) * 2.0 / 3.0
return ret
def delta(lat, lon):
'''
Krasovsky 1940
//
// a = 6378245.0, 1/f = 298.3
// b = a * (1 - f)
// ee = (a^2 - b^2) / a^2;
'''
a = 6378245.0 # a: Projection factor of satellite ellipsoidal coordinates projected onto a flat map coordinate system
ee = 0.00669342162296594323 # ee: Eccentricity of ellipsoid
dLat = transformLat(lon - 105.0, lat - 35.0)
dLon = transformLon(lon - 105.0, lat - 35.0)
radLat = lat / 180.0 * math.pi
magic = math.sin(radLat)
magic = 1 - ee * magic * magic
sqrtMagic = math.sqrt(magic)
dLat = (dLat * 180.0) / ((a * (1 - ee)) / (magic * sqrtMagic) * math.pi)
dLon = (dLon * 180.0) / (a / sqrtMagic * math.cos(radLat) * math.pi)
return {'lat': dLat, 'lon': dLon}
def outOfChina(lat, lon):
if (lon < 72.004 or lon > 137.8347):
return True
if (lat < 0.8293 or lat > 55.8271):
return True
return False
def gcj_to_wgs(gcjLon, gcjLat):
if outOfChina(gcjLat, gcjLon):
return (gcjLon, gcjLat)
d = delta(gcjLat, gcjLon)
return (gcjLon - d["lon"], gcjLat - d["lat"])
def wgs_to_gcj(wgsLon, wgsLat):
if outOfChina(wgsLat, wgsLon):
return wgsLon, wgsLat
d = delta(wgsLat, wgsLon);
return wgsLon + d["lon"], wgsLat + d["lat"]
# --------------------------------------------------------------
# ---------------------------------------------------------
# Get tile coordinates in Google Maps based on latitude and longitude of WGS-84
def wgs_to_tile(j, w, z):
'''
Get google-style tile cooridinate from geographical coordinate
j : Longittude
w : Latitude
z : zoom
'''
isnum = lambda x: isinstance(x, int) or isinstance(x, float)
if not (isnum(j) and isnum(w)):
raise TypeError("j and w must be int or float!")
if not isinstance(z, int) or z < 0 or z > 22:
raise TypeError("z must be int and between 0 to 22.")
if j < 0:
j = 180 + j
else:
j += 180
j /= 360 # make j to (0,1)
w = 85.0511287798 if w > 85.0511287798 else w
w = -85.0511287798 if w < -85.0511287798 else w
w = log(tan((90 + w) * pi / 360)) / (pi / 180)
w /= 180 # make w to (-1,1)
w = 1 - (w + 1) / 2 # make w to (0,1) and left top is 0-point
num = 2 ** z
x = floor(j * num)
y = floor(w * num)
return x, y
def pixls_to_mercator(zb):
# Get the web Mercator projection coordinates of the four corners of the area according to the four corner coordinates of the tile
inx, iny = zb["LT"] # left top
inx2, iny2 = zb["RB"] # right bottom
length = 20037508.3427892
sum = 2 ** zb["z"]
LTx = inx / sum * length * 2 - length
LTy = -(iny / sum * length * 2) + length
RBx = (inx2 + 1) / sum * length * 2 - length
RBy = -((iny2 + 1) / sum * length * 2) + length
# LT=left top,RB=right buttom
# Returns the projected coordinates of the four corners
res = {'LT': (LTx, LTy), 'RB': (RBx, RBy),
'LB': (LTx, RBy), 'RT': (RBx, LTy)}
return res
def tile_to_pixls(zb):
# Tile coordinates are converted to pixel coordinates of the four corners
out = {}
width = (zb["RT"][0] - zb["LT"][0] + 1) * 256
height = (zb["LB"][1] - zb["LT"][1] + 1) * 256
out["LT"] = (0, 0)
out["RT"] = (width, 0)
out["LB"] = (0, -height)
out["RB"] = (width, -height)
return out
# -----------------------------------------------------------
# ---------------------------------------------------------
class Downloader(Thread):
# multiple threads downloader
def __init__(self, index, count, urls, datas, update):
# index represents the number of threads
# count represents the total number of threads
# urls represents the list of URLs nedd to be downloaded
# datas represents the list of data need to be returned.
super().__init__()
self.urls = urls
self.datas = datas
self.index = index
self.count = count
self.update = update
def download(self, url):
HEADERS = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36 Edg/88.0.705.68'}
header = ur.Request(url, headers=HEADERS)
err = 0
while (err < 3):
try:
data = ur.urlopen(header).read()
except:
err += 1
else:
return data
raise Exception("Bad network link.")
def run(self):
for i, url in enumerate(self.urls):
if i % self.count != self.index:
continue
self.datas[i] = self.download(url)
if mutex.acquire():
self.update()
mutex.release()
# ---------------------------------------------------------
# ---------------------------------------------------------
MAP_URLS = {
"Google": "http://mts0.googleapis.com/vt?lyrs={style}&x={x}&y={y}&z={z}",
"Google China": "http://mt2.google.cn/vt/lyrs={style}&hl=zh-CN&gl=CN&src=app&x={x}&y={y}&z={z}"}
def get_url(source, x, y, z, style): #
if source == 'Google China':
url = MAP_URLS["Google China"].format(x=x, y=y, z=z, style=style)
elif source == 'Google':
url = MAP_URLS["Google"].format(x=x, y=y, z=z, style=style)
else:
raise Exception("Unknown Map Source ! ")
return url
def get_urls(x1, y1, x2, y2, z, source='google', style='s'):
pos1x, pos1y = wgs_to_tile(x1, y1, z)
pos2x, pos2y = wgs_to_tile(x2, y2, z)
lenx = pos2x - pos1x + 1
leny = pos2y - pos1y + 1
print("Total tiles number:{x} X {y}".format(x=lenx, y=leny))
urls = [get_url(source, i, j, z, style) for j in range(pos1y, pos1y + leny) for i in range(pos1x, pos1x + lenx)]
return urls
# ---------------------------------------------------------
# ---------------------------------------------------------
def download_tiles(urls, multi=10):
def makeupdate(s):
def up():
global COUNT
COUNT += 1
print("\rDownLoading...[{0}/{1}]".format(COUNT, s), end='')
return up
url_len = len(urls)
datas = [None] * url_len
if multi < 1 or multi > 20 or not isinstance(multi, int):
raise Exception("multi of Downloader shuold be int and between 1 to 20.")
tasks = [Downloader(i, multi, urls, datas, makeupdate(url_len)) for i in range(multi)]
for i in tasks:
i.start()
for i in tasks:
i.join()
return datas
def merge_tiles(datas, x1, y1, x2, y2, z):
pos1x, pos1y = wgs_to_tile(x1, y1, z)
pos2x, pos2y = wgs_to_tile(x2, y2, z)
lenx = pos2x - pos1x + 1
leny = pos2y - pos1y + 1
outpic = pil.new('RGBA', (lenx * 256, leny * 256))
for i, data in enumerate(datas):
picio = io.BytesIO(data)
small_pic = pil.open(picio)
y, x = i // lenx, i % lenx
outpic.paste(small_pic, (x * 256, y * 256))
print('\nTiles merge completed')
return outpic
# ---------------------------------------------------------
# ---------------------------------------------------------
def getExtent(x1, y1, x2, y2, z, source="Google China"):
pos1x, pos1y = wgs_to_tile(x1, y1, z)
pos2x, pos2y = wgs_to_tile(x2, y2, z)
Xframe = pixls_to_mercator(
{"LT": (pos1x, pos1y), "RT": (pos2x, pos1y), "LB": (pos1x, pos2y), "RB": (pos2x, pos2y), "z": z})
for i in ["LT", "LB", "RT", "RB"]:
Xframe[i] = mercator_to_wgs(*Xframe[i])
if source == "Google":
pass
elif source == "Google China":
for i in ["LT", "LB", "RT", "RB"]:
Xframe[i] = gcj_to_wgs(*Xframe[i])
else:
raise Exception("Invalid argument: source.")
return Xframe
def saveTiff(r, g, b, gt, filePath):
fname_out = filePath
driver = gdal.GetDriverByName('GTiff')
# Create a 3-band dataset
dset_output = driver.Create(fname_out, r.shape[1], r.shape[0], 3, gdal.GDT_Byte)
dset_output.SetGeoTransform(gt)
try:
proj = osr.SpatialReference()
proj.ImportFromEPSG(4326)
dset_output.SetSpatialRef(proj)
except:
print("Error: Coordinate system setting failed")
dset_output.GetRasterBand(1).WriteArray(r)
dset_output.GetRasterBand(2).WriteArray(g)
dset_output.GetRasterBand(3).WriteArray(b)
dset_output.FlushCache()
dset_output = None
print("Image Saved")
# ---------------------------------------------------------
def main(left, top, right, bottom, zoom, filePath, style='s', server="Google China"):
"""
Download images based on spatial extent.
East longitude is positive and west longitude is negative.
North latitude is positive, south latitude is negative.
Parameters
----------
left, top : left-top coordinate, for example (100.361,38.866)
right, bottom : right-bottom coordinate
z : zoom
filePath : File path for storing results, TIFF format
style :
m for map;
s for satellite;
y for satellite with label;
t for terrain;
p for terrain with label;
h for label;
source : Google China (default) or Google
"""
# ---------------------------------------------------------
# Get the urls of all tiles in the extent
urls = get_urls(left, top, right, bottom, zoom, server, style)
# Download tiles
datas = download_tiles(urls)
# Combine downloaded tile maps into one map
outpic = merge_tiles(datas, left, top, right, bottom, zoom)
outpic = outpic.convert('RGB')
r, g, b = cv2.split(np.array(outpic))
# Get the spatial information of the four corners of the merged map and use it for outputting
extent = getExtent(left, top, right, bottom, zoom, server)
gt = (extent['LT'][0], (extent['RB'][0] - extent['LT'][0]) / r.shape[1], 0, extent['LT'][1], 0,
(extent['RB'][1] - extent['LT'][1]) / r.shape[0])
saveTiff(r, g, b, gt, filePath)
# ---------------------------------------------------------
if __name__ == '__main__':
start_time = time.time()
COUNT = 0 # Progress display, starting at 0
mutex = Lock()
main(100.361, 38.866, 100.386, 38.839, 13, r'D:\Documents\Temp\test1.tif', server="Google")
end_time = time.time()
print('lasted a total of {:.2f} seconds'.format(end_time - start_time))
================================================
FILE: downloader_1.2.py
================================================
# -*- coding: utf-8 -*
'''
This code is used to download image from google
@date : 2020-3-13
@author: Zheng Jie
@E-mail: zhengjie9510@qq.com
'''
import io
import math
import multiprocessing
import time
import urllib.request as ur
from math import floor, pi, log, tan, atan, exp
from threading import Thread
import PIL.Image as pil
import cv2
import numpy as np
from osgeo import gdal, osr
# ------------------Interchange between WGS-84 and Web Mercator-------------------------
# WGS-84 to Web Mercator
def wgs_to_mercator(x, y):
y = 85.0511287798 if y > 85.0511287798 else y
y = -85.0511287798 if y < -85.0511287798 else y
x2 = x * 20037508.34 / 180
y2 = log(tan((90 + y) * pi / 360)) / (pi / 180)
y2 = y2 * 20037508.34 / 180
return x2, y2
# Web Mercator to WGS-84
def mercator_to_wgs(x, y):
x2 = x / 20037508.34 * 180
y2 = y / 20037508.34 * 180
y2 = 180 / pi * (2 * atan(exp(y2 * pi / 180)) - pi / 2)
return x2, y2
# --------------------------------------------------------------------------------------
# -----------------Interchange between GCJ-02 to WGS-84---------------------------
# All public geographic data in mainland China need to be encrypted with GCJ-02, introducing random bias
# This part of the code is used to remove the bias
def transformLat(x, y):
ret = -100.0 + 2.0 * x + 3.0 * y + 0.2 * y * y + 0.1 * x * y + 0.2 * math.sqrt(abs(x))
ret += (20.0 * math.sin(6.0 * x * math.pi) + 20.0 * math.sin(2.0 * x * math.pi)) * 2.0 / 3.0
ret += (20.0 * math.sin(y * math.pi) + 40.0 * math.sin(y / 3.0 * math.pi)) * 2.0 / 3.0
ret += (160.0 * math.sin(y / 12.0 * math.pi) + 320 * math.sin(y * math.pi / 30.0)) * 2.0 / 3.0
return ret
def transformLon(x, y):
ret = 300.0 + x + 2.0 * y + 0.1 * x * x + 0.1 * x * y + 0.1 * math.sqrt(abs(x))
ret += (20.0 * math.sin(6.0 * x * math.pi) + 20.0 * math.sin(2.0 * x * math.pi)) * 2.0 / 3.0
ret += (20.0 * math.sin(x * math.pi) + 40.0 * math.sin(x / 3.0 * math.pi)) * 2.0 / 3.0
ret += (150.0 * math.sin(x / 12.0 * math.pi) + 300.0 * math.sin(x / 30.0 * math.pi)) * 2.0 / 3.0
return ret
def delta(lat, lon):
'''
Krasovsky 1940
//
// a = 6378245.0, 1/f = 298.3
// b = a * (1 - f)
// ee = (a^2 - b^2) / a^2;
'''
a = 6378245.0 # a: Projection factor of satellite ellipsoidal coordinates projected onto a flat map coordinate system
ee = 0.00669342162296594323 # ee: Eccentricity of ellipsoid
dLat = transformLat(lon - 105.0, lat - 35.0)
dLon = transformLon(lon - 105.0, lat - 35.0)
radLat = lat / 180.0 * math.pi
magic = math.sin(radLat)
magic = 1 - ee * magic * magic
sqrtMagic = math.sqrt(magic)
dLat = (dLat * 180.0) / ((a * (1 - ee)) / (magic * sqrtMagic) * math.pi)
dLon = (dLon * 180.0) / (a / sqrtMagic * math.cos(radLat) * math.pi)
return {'lat': dLat, 'lon': dLon}
def outOfChina(lat, lon):
if (lon < 72.004 or lon > 137.8347):
return True
if (lat < 0.8293 or lat > 55.8271):
return True
return False
def gcj_to_wgs(gcjLon, gcjLat):
if outOfChina(gcjLat, gcjLon):
return (gcjLon, gcjLat)
d = delta(gcjLat, gcjLon)
return (gcjLon - d["lon"], gcjLat - d["lat"])
def wgs_to_gcj(wgsLon, wgsLat):
if outOfChina(wgsLat, wgsLon):
return wgsLon, wgsLat
d = delta(wgsLat, wgsLon)
return wgsLon + d["lon"], wgsLat + d["lat"]
# --------------------------------------------------------------
# ---------------------------------------------------------
# Get tile coordinates in Google Maps based on latitude and longitude of WGS-84
def wgs_to_tile(j, w, z):
'''
Get google-style tile cooridinate from geographical coordinate
j : Longittude
w : Latitude
z : zoom
'''
isnum = lambda x: isinstance(x, int) or isinstance(x, float)
if not (isnum(j) and isnum(w)):
raise TypeError("j and w must be int or float!")
if not isinstance(z, int) or z < 0 or z > 22:
raise TypeError("z must be int and between 0 to 22.")
if j < 0:
j = 180 + j
else:
j += 180
j /= 360 # make j to (0,1)
w = 85.0511287798 if w > 85.0511287798 else w
w = -85.0511287798 if w < -85.0511287798 else w
w = log(tan((90 + w) * pi / 360)) / (pi / 180)
w /= 180 # make w to (-1,1)
w = 1 - (w + 1) / 2 # make w to (0,1) and left top is 0-point
num = 2 ** z
x = floor(j * num)
y = floor(w * num)
return x, y
def pixls_to_mercator(zb):
# Get the web Mercator projection coordinates of the four corners of the area according to the four corner coordinates of the tile
inx, iny = zb["LT"] # left top
inx2, iny2 = zb["RB"] # right bottom
length = 20037508.3427892
sum = 2 ** zb["z"]
LTx = inx / sum * length * 2 - length
LTy = -(iny / sum * length * 2) + length
RBx = (inx2 + 1) / sum * length * 2 - length
RBy = -((iny2 + 1) / sum * length * 2) + length
# LT=left top,RB=right buttom
# Returns the projected coordinates of the four corners
res = {'LT': (LTx, LTy), 'RB': (RBx, RBy),
'LB': (LTx, RBy), 'RT': (RBx, LTy)}
return res
def tile_to_pixls(zb):
# Tile coordinates are converted to pixel coordinates of the four corners
out = {}
width = (zb["RT"][0] - zb["LT"][0] + 1) * 256
height = (zb["LB"][1] - zb["LT"][1] + 1) * 256
out["LT"] = (0, 0)
out["RT"] = (width, 0)
out["LB"] = (0, -height)
out["RB"] = (width, -height)
return out
# -----------------------------------------------------------
# ---------------------------------------------------------
class Downloader(Thread):
# multiple threads downloader
def __init__(self, index, count, urls, datas):
# index represents the number of threads
# count represents the total number of threads
# urls represents the list of URLs nedd to be downloaded
# datas represents the list of data need to be returned.
super().__init__()
self.urls = urls
self.datas = datas
self.index = index
self.count = count
def download(self, url):
HEADERS = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36 Edg/88.0.705.68'}
header = ur.Request(url, headers=HEADERS)
err = 0
while (err < 3):
try:
data = ur.urlopen(header).read()
except:
err += 1
else:
return data
raise Exception("Bad network link.")
def run(self):
for i, url in enumerate(self.urls):
if i % self.count != self.index:
continue
self.datas[i] = self.download(url)
# ---------------------------------------------------------
# ---------------------------------------------------------
def getExtent(x1, y1, x2, y2, z, source="Google China"):
pos1x, pos1y = wgs_to_tile(x1, y1, z)
pos2x, pos2y = wgs_to_tile(x2, y2, z)
Xframe = pixls_to_mercator(
{"LT": (pos1x, pos1y), "RT": (pos2x, pos1y), "LB": (pos1x, pos2y), "RB": (pos2x, pos2y), "z": z})
for i in ["LT", "LB", "RT", "RB"]:
Xframe[i] = mercator_to_wgs(*Xframe[i])
if source == "Google":
pass
elif source == "Google China":
for i in ["LT", "LB", "RT", "RB"]:
Xframe[i] = gcj_to_wgs(*Xframe[i])
else:
raise Exception("Invalid argument: source.")
return Xframe
def saveTiff(r, g, b, gt, filePath):
fname_out = filePath
driver = gdal.GetDriverByName('GTiff')
# Create a 3-band dataset
dset_output = driver.Create(fname_out, r.shape[1], r.shape[0], 3, gdal.GDT_Byte)
dset_output.SetGeoTransform(gt)
try:
proj = osr.SpatialReference()
proj.ImportFromEPSG(4326)
dset_output.SetSpatialRef(proj)
except:
print("Error: Coordinate system setting failed")
dset_output.GetRasterBand(1).WriteArray(r)
dset_output.GetRasterBand(2).WriteArray(g)
dset_output.GetRasterBand(3).WriteArray(b)
dset_output.FlushCache()
dset_output = None
print("Image Saved")
# ---------------------------------------------------------
# ---------------------------------------------------------
MAP_URLS = {
"Google": "http://mts0.googleapis.com/vt?lyrs={style}&x={x}&y={y}&z={z}",
"Google China": "http://mt2.google.cn/vt/lyrs={style}&hl=zh-CN&gl=CN&src=app&x={x}&y={y}&z={z}"}
def get_url(source, x, y, z, style): #
if source == 'Google China':
url = MAP_URLS["Google China"].format(x=x, y=y, z=z, style=style)
elif source == 'Google':
url = MAP_URLS["Google"].format(x=x, y=y, z=z, style=style)
else:
raise Exception("Unknown Map Source ! ")
return url
def get_urls(x1, y1, x2, y2, z, source, style):
pos1x, pos1y = wgs_to_tile(x1, y1, z)
pos2x, pos2y = wgs_to_tile(x2, y2, z)
lenx = pos2x - pos1x + 1
leny = pos2y - pos1y + 1
print("Total tiles number:{x} X {y}".format(x=lenx, y=leny))
urls = [get_url(source, i, j, z, style) for j in range(pos1y, pos1y + leny) for i in range(pos1x, pos1x + lenx)]
return urls
# ---------------------------------------------------------
# ---------------------------------------------------------
def merge_tiles(datas, x1, y1, x2, y2, z):
pos1x, pos1y = wgs_to_tile(x1, y1, z)
pos2x, pos2y = wgs_to_tile(x2, y2, z)
lenx = pos2x - pos1x + 1
leny = pos2y - pos1y + 1
outpic = pil.new('RGBA', (lenx * 256, leny * 256))
for i, data in enumerate(datas):
picio = io.BytesIO(data)
small_pic = pil.open(picio)
y, x = i // lenx, i % lenx
outpic.paste(small_pic, (x * 256, y * 256))
print('Tiles merge completed')
return outpic
def download_tiles(urls, multi=10):
url_len = len(urls)
datas = [None] * url_len
if multi < 1 or multi > 20 or not isinstance(multi, int):
raise Exception("multi of Downloader shuold be int and between 1 to 20.")
tasks = [Downloader(i, multi, urls, datas) for i in range(multi)]
for i in tasks:
i.start()
for i in tasks:
i.join()
return datas
# ---------------------------------------------------------
# ---------------------------------------------------------
def main(left, top, right, bottom, zoom, filePath, style='s', server="Google China"):
"""
Download images based on spatial extent.
East longitude is positive and west longitude is negative.
North latitude is positive, south latitude is negative.
Parameters
----------
left, top : left-top coordinate, for example (100.361,38.866)
right, bottom : right-bottom coordinate
z : zoom
filePath : File path for storing results, TIFF format
style :
m for map;
s for satellite;
y for satellite with label;
t for terrain;
p for terrain with label;
h for label;
source : Google China (default) or Google
"""
# ---------------------------------------------------------
# Get the urls of all tiles in the extent
urls = get_urls(left, top, right, bottom, zoom, server, style)
# Group URLs based on the number of CPU cores to achieve roughly equal amounts of tasks
urls_group = [urls[i:i + math.ceil(len(urls) / multiprocessing.cpu_count())] for i in
range(0, len(urls), math.ceil(len(urls) / multiprocessing.cpu_count()))]
# Each set of URLs corresponds to a process for downloading tile maps
print('Tiles downloading......')
pool = multiprocessing.Pool(multiprocessing.cpu_count())
results = pool.map(download_tiles, urls_group)
pool.close()
pool.join()
result = [x for j in results for x in j]
print('Tiles download complete')
# Combine downloaded tile maps into one map
outpic = merge_tiles(result, left, top, right, bottom, zoom)
outpic = outpic.convert('RGB')
r, g, b = cv2.split(np.array(outpic))
# Get the spatial information of the four corners of the merged map and use it for outputting
extent = getExtent(left, top, right, bottom, zoom, server)
gt = (extent['LT'][0], (extent['RB'][0] - extent['LT'][0]) / r.shape[1], 0, extent['LT'][1], 0,
(extent['RB'][1] - extent['LT'][1]) / r.shape[0])
saveTiff(r, g, b, gt, filePath)
# ---------------------------------------------------------
if __name__ == '__main__':
start_time = time.time()
main(100.361, 38.866, 100.386, 38.839, 13, r'D:\Documents\Temp\test.tif', server="Google")
end_time = time.time()
print('lasted a total of {:.2f} seconds'.format(end_time - start_time))
================================================
FILE: requirements.txt
================================================
GDAL==3.2.1
numpy==1.20.1
Pillow==10.0.1
py-opencv==4.5.1
gitextract_hk_bi296/ ├── LICENSE ├── README.md ├── downloader_1.1.py ├── downloader_1.2.py └── requirements.txt
SYMBOL INDEX (44 symbols across 2 files)
FILE: downloader_1.1.py
function wgs_to_mercator (line 25) | def wgs_to_mercator(x, y):
function mercator_to_wgs (line 36) | def mercator_to_wgs(x, y):
function transformLat (line 48) | def transformLat(x, y):
function transformLon (line 56) | def transformLon(x, y):
function delta (line 64) | def delta(lat, lon):
function outOfChina (line 85) | def outOfChina(lat, lon):
function gcj_to_wgs (line 93) | def gcj_to_wgs(gcjLon, gcjLat):
function wgs_to_gcj (line 100) | def wgs_to_gcj(wgsLon, wgsLat):
function wgs_to_tile (line 111) | def wgs_to_tile(j, w, z):
function pixls_to_mercator (line 143) | def pixls_to_mercator(zb):
function tile_to_pixls (line 162) | def tile_to_pixls(zb):
class Downloader (line 177) | class Downloader(Thread):
method __init__ (line 179) | def __init__(self, index, count, urls, datas, update):
method download (line 191) | def download(self, url):
method run (line 205) | def run(self):
function get_url (line 223) | def get_url(source, x, y, z, style): #
function get_urls (line 233) | def get_urls(x1, y1, x2, y2, z, source='google', style='s'):
function download_tiles (line 246) | def download_tiles(urls, multi=10):
function merge_tiles (line 267) | def merge_tiles(datas, x1, y1, x2, y2, z):
function getExtent (line 286) | def getExtent(x1, y1, x2, y2, z, source="Google China"):
function saveTiff (line 303) | def saveTiff(r, g, b, gt, filePath):
function main (line 325) | def main(left, top, right, bottom, zoom, filePath, style='s', server="Go...
FILE: downloader_1.2.py
function wgs_to_mercator (line 26) | def wgs_to_mercator(x, y):
function mercator_to_wgs (line 37) | def mercator_to_wgs(x, y):
function transformLat (line 49) | def transformLat(x, y):
function transformLon (line 57) | def transformLon(x, y):
function delta (line 65) | def delta(lat, lon):
function outOfChina (line 86) | def outOfChina(lat, lon):
function gcj_to_wgs (line 94) | def gcj_to_wgs(gcjLon, gcjLat):
function wgs_to_gcj (line 101) | def wgs_to_gcj(wgsLon, wgsLat):
function wgs_to_tile (line 112) | def wgs_to_tile(j, w, z):
function pixls_to_mercator (line 144) | def pixls_to_mercator(zb):
function tile_to_pixls (line 163) | def tile_to_pixls(zb):
class Downloader (line 178) | class Downloader(Thread):
method __init__ (line 180) | def __init__(self, index, count, urls, datas):
method download (line 191) | def download(self, url):
method run (line 205) | def run(self):
function getExtent (line 215) | def getExtent(x1, y1, x2, y2, z, source="Google China"):
function saveTiff (line 232) | def saveTiff(r, g, b, gt, filePath):
function get_url (line 260) | def get_url(source, x, y, z, style): #
function get_urls (line 270) | def get_urls(x1, y1, x2, y2, z, source, style):
function merge_tiles (line 283) | def merge_tiles(datas, x1, y1, x2, y2, z):
function download_tiles (line 298) | def download_tiles(urls, multi=10):
function main (line 314) | def main(left, top, right, bottom, zoom, filePath, style='s', server="Go...
Condensed preview — 5 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (31K chars).
[
{
"path": "LICENSE",
"chars": 1069,
"preview": "MIT License\n\nCopyright (c) 2020 zhengjie9510\n\nPermission is hereby granted, free of charge, to any person obtaining a co"
},
{
"path": "README.md",
"chars": 2871,
"preview": "# Google-Map-Downloader\n. The extraction includes 5 files (28.6 KB), approximately 9.1k tokens, and a symbol index with 44 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.