Amazon has launched Earth on AWS, a new landing page that highlights large geospatial datasets. These are made openly available in the cloud as AWS Public Dataset. There are also resources to learn how to build planetary-scale applications in the cloud with open geospatial data.
In addition to previously launched, Landsat, NEXRAD, and SpaceNet datasets, new datasets made up of high resolution aerial imagery, global event information, and global elevation data are now available.
The current databases are:
Landsat on AWS
Landsat 8 data is available for anyone to use via Amazon S3. All Landsat 8 scenes from 2015 are available along with a selection of cloud-free scenes from 2013 and 2014. All new Landsat 8 scenes are made available each day, often within hours of production. The satellite images the entire Earth every 16 days at a roughly 30 meter resolution.
NEXRAD on AWS
The Next Generation Weather Radar (NEXRAD) is a network of 160 high-resolution Doppler radar sites that detects precipitation and atmospheric movement and disseminates data in approximately five minute intervals from each site. NEXRAD enables severe storm prediction and is used by researchers and commercial enterprises to study and address the impact of weather across multiple sectors.
SpaceNet Machine Learning Imagery
SpaceNet is a corpus of high-resolution satellite imagery and labeled training data for researchers to utilize to develop and train machine learning algorithms. The data set is made up of roughly 1,990 square kilometers of imagery at 50 cm resolution and 220,594 corresponding building footprints.
National Agriculture Imagery Program
The National Agriculture Imagery Program (NAIP) acquires aerial imagery during the agricultural growing seasons in the continental U.S. Roughly one meter aerial imagery (Red, Green, Blue, NIR) is available for multiple years on Amazon S3.
Terrain Tiles on AWS
Digital Elevation Models (DEM) provide a way to examine the elevation of the Earth's surface and are available as terrain raster tiles on Amazon S3. Global coverage is provided and in the United States, 10 meter National Elevation Data (NED) data now augments the earlier NED 3 meter and 30 meter Shuttle Radar Topography Mission (SRTM) data for crisper, more consistent mountain detail.
GDELT - A Global Database of Society
The GDELT Project monitors the world's broadcast, print, and web news from nearly every corner of every country in over 100 languages and identifies the people, locations, organizations, counts, themes, sources, emotions, counts, quotes, images and events driving our global society every second of every day.
NASA Earth Exchange (NEX)
The NASA Earth Exchange (NEX) makes it easier and more efficient for researchers to access and process earth science data. NEX datasets available on Amazon S3 include downscaled climate projections, global MODIS vegetation indices, and Landsat Global Land Survey data.
GSOD - Daily Global Weather Measurements
Global Surface Summary of Day (GSOD) is a collection of daily weather measurements (temperature, wind speed, humidity, pressure, and more) from 9000+ weather stations around the world. CSVs for measurements starting in 1929 are available on Amazon S3.
Amazon are also opening a call for proposals from researchers to receive promotional Cloud Credits for Research to work with Earth observation data on AWS. Learn more at https://aws.amazon.com/earth.
We all know that the Travelling Salesman Problem is NP complete so how do you tackle the very difficult problem? The Google Research team has provided an account of how they went about solving this optimization:
"Our team in Google Research has been fascinated by the “Geometry of Place” for some time, and we started investigating a question related to Euler’s: rather than visiting just the bridges, how can we visit as many interesting places as possible during a particular trip? We call this the “itineraries” problem. Euler didn’t study it, but it is a well known topic in Optimization, where it is often called the “Orienteering” problem.
While Euler’s problem has an efficient and exact solution, the itineraries problem is not just hard to solve, it is hard to even approximately solve! The difficulty lies in the interplay between two conflicting goals: first, we should pick great places to visit, but second, we should pick them to allow a good itinerary: not too much travel time; don’t visit places when they’re closed; don’t visit too many museums, etc. Embedded in such problems is the challenge of finding efficient routes, often referred to as the Travelling Salesman Problem (TSP)....
....While the algorithmic aspects of the problem were highly challenging, we realized that producing high-quality itineraries was just as dependent on our understanding of the many possible stopping points on the itinerary. We had Google’s extensive travel database to identify the interesting places to visit, and we also had great data from Google’s existing systems about how to travel from any place to any other. But we didn’t have a good sense for how people typically move through this geometry of places.
For this, we turned to the wisdom of crowds. This type of wisdom is used by Google to estimate delays on highways, and to discover when restaurants are most busy. Here, we use the same techniques to learn about common visit sequences that we can stitch together into itineraries that feel good to our users. We combine Google's knowledge of when places are popular, with the directions between those places to gather an idea of what tourists like to do when travelling.
And the crowd has a lot more wisdom to offer in the future. For example, we noticed that visits to Buckingham Palace spike around 11:30 and stay a bit longer than at other times of the day. This seemed a little strange to us, but when we looked more closely, it turns out to be the time of theChanging of the Guard. We’re looking now at ways to incorporate this type of timing information into the itinerary selection algorithms."
So give it a try: Google Trips, available now on Android and iOS, has you covered from departure to return.
This year's premium conference for PostgreSQL took place in Athens, Greece between October 22-25. The nice Athenian weather and cultural aspect aside, the conference was a big hit too.
Are you ready for Thanksgiving, when overeating remorse and a surfeit of being thankful causes the unsettling thought that there are only four weeks till the Xmas break? So here is a mix of weird [ ... ]
