Posts

Maps in the form of digital images are widely available in geoportals, Web pages, and other data sources. The metadata of map images, such as spatial extents and place names, are critical for their indexing and searching. However, many map images have either mismatched metadata or no metadata at all. Recent developments in deep learning offer new possibilities for enriching the metadata of map images via image-based information extraction. One major challenge of using deep learning models is that they often require large amounts of training data that have to be manually labeled. To address this challenge, this paper presents a deep learning approach with GIS-based data augmentation that can automatically generate labeled training map images from shapefiles using GIS operations. We utilize such an approach to enrich the metadata of map images by adding spatial extents and place names extracted from map images. We evaluate this GIS-based data augmentation approach by using it to train multiple deep learning models and testing them on two different datasets: a Web Map Service image dataset at the continental scale and an online map image dataset at the state scale. We then discuss the advantages and limitations of the proposed approach.

More details are available in our paper at:
Yingjie Hu, Zhipeng Gui, Jimin Wang, and Muxian Li (2021): Enriching the metadata of map images: a deep learning approach with GIS-based data augmentation. International Journal of Geographical Information Science, in press. [PDF]

The University at Buffalo Artificial Intelligence Institute was established in 2018 to explore ways to combine machines’ superior ability to ingest, connect and recall information with concepts that humans excel at, such as reasoning, judgement and strategizing, to develop dynamic human-machine partnerships. Its mission is to bring together educators and researchers in an interdisciplinary environment to continue to make significant breakthroughs in advancing the promise of machine or human-machine systems that can address complex cognitive tasks.

Our GeoAI Lab was invited to join UB AI Institute as an affiliated lab. This affiliation will further strengthen the connections and collaborations between our lab and other UB units.

The objective of this project is to understand how people describe locations on social media during natural disasters. These data from social media are potentially beneficial in disaster response efforts, and to further this goal, computational algorithms are being developed to extract location information from social media postings. However, uneven use of social media by different populations and varying ways of describing places can complicate the process of identifying locations. This project advances knowledge by enhancing the understanding of the ways in which people describe geographic locations during natural disasters, the effectiveness of different algorithmic approaches for location extraction, and the potential spatial biases in the described locations. Such knowledge benefits society by informing future disaster response practices to help save lives and reduce inequality in response efforts. This project provides interdisciplinary research experience for undergraduates and graduates and will enhance academia and industry partnership. The datasets and algorithmic tools produced from this project will be publicly shared.

Social media platforms, such as Twitter, are increasingly being used by people impacted by natural disasters. Descriptions about the locations of victims and accidents are often contained in help-seeking messages posted on these platforms. However, a limited understanding exists of how locations are described on social media during natural disasters, which hinders their automatic extraction via computational tools. This project addresses three research questions: (1) What are the typical forms of location descriptions used by people on social media during natural disasters? (2) How effective are different geospatial artificial intelligence (GeoAI) approaches for extracting these location descriptions and representing them in geographic space? And (3) What spatial biases have characterized location descriptions on social media during natural disasters? The research team is collaborating with emergency management specialists to understand location descriptions on social media, examine multiple geo-knowledge-informed AI approaches for location extraction, and investigate the spatial biases of the extracted locations and their relation to vulnerable communities. The obtained knowledge about location descriptions and the developed methods can be applied to future disasters in diverse settings.

NSF award page: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2117771

We are seeking three UB undergraduate students with work-study support to contribute to two NASA-funded projects in collaboration with an interdisciplinary team of scientists and conservation professionals located in Buffalo, NY, Merced, CA, and Cape Town, South Africa.

Scientific Illustrator – Are you interested in science but passionate about art? Create digital artwork that captures the incredible biodiversity of South Africa and the technology we are using to study it. Learn more and apply at https://app.joinhandshake.com/jobs/4834211.

Geographic Data Visualization Analyst – Are you interested in learning how to tell stories with maps and building your online GIS portfolio? Join us to develop ‘story maps’ that share geospatial information, photographs, and details about the biodiversity hotspot and project. Learn more and apply at https://app.joinhandshake.com/jobs/4834301.

GIS Analyst – Do you want to get involved under the hood of a NASA project using high-resolution imagery and artificial intelligence to help conserve biodiversity? Help us document ecological change by developing a geospatial dataset we’ll use for model training and prediction. Learn more and apply at https://app.joinhandshake.com/jobs/4838720.

Important – these positions are only open to current UB students with work-study support for the 2021-2022 academic year. See here for more information about work-study positions at UB.

We received the notification from NASA that our project proposal “Near-Real-Time Forecasting and Change Detection for a Fire-Prone Shrubland Ecosystem” was selected for funding support. This project aims to utilize statistical modeling and GeoAI methods for near-term ecological forecasting to predict natural land surface processes and evaluate near-real-time changes in the state of a hyperdiverse, fire-dependent and seasonally fluctuating open ecosystem: the fynbos of the Cape Floristic Region (CFR) of South Africa.

Our research team consists of:
• Dr. Adam M. Wilson, Principal Investigator, Wilson Lab, Department of Geography, University at Buffalo, State University of New York, United States
• Dr. Yingjie Hu, Co-Investigator, GeoAI Lab, Department of Geography, University at Buffalo, State University of New York, United States
• Dr. Glenn R. Moncrieff, Co-Investigator, Fynbos Node, South African Environmental Observation Network, South Africa
• Dr. Jasper A. Slingsby, Co-Investigator, Fynbos Node, South African Environmental Observation Network, South Africa

This project further enhances our current AI for Earth project funded by Microsoft and led by Dr. Hu.

We will be hiring a Graduate Research Assistant (starting from Fall 2022 or earlier) and a post doc researcher (starting around Summer 2021), both of whom will be co-advised by Dr. Adam Wilson and Dr. Yingjie Hu. By participating in this project, the GRA and post doc researchers will develop expertise on GeoAI, raster data processing, biodiversity, and ecological forecasting. Interested candidates are encouraged to contact Dr. Hu.

The pandemic made it more difficult for many of us to do research. Thanks to the initiative of AAG, faculty members throughout the country were brought together to share their expertise and help students during this challenging time! I (Yingjie) had a great time leading the workshop on “Integrating Machine Learning into Geographic Research” in the past week. While the AAG committee and I initially planned only a 20-person small workshop to ensure personal level interactions, this workshop received 175 registrations from not only students but also researchers all over the world (see the map of the registrants).

Map of the workshop registrants.

This overwhelming interest is a nice surprise to us, but it also means that we would have to reject a large number of students and researchers who are eager to learn, if we were just to admit 20 participants. Meanwhile, having more participants in the workshop will make it more difficult for the students to have personal interaction with the instructors. Eventually, we admitted 21 students as “active participants” who can actively engage in the workshop, while a large number of other registrants were admitted as “observers” who can still call in and listen to the workshop lectures.

The workshop was a great experience, as I can share my knowledge on GIS and Machine Learning with a wide audience while interacting with each individual active participant. Thank you all for your participating, and many thanks to our AAG workshop committee, particularly Julaiti and Coline for their great help and support!

With the many challenges posed by the pandemic of COVID-19, the American Association of Geographers (AAG) called upon geography faculty members throughout the nation to help offer a series of virtual workshops and seminars (for AAG members only) to support graduate students adapt in their research. Dr. Hu is one of these selected faculty members.

Our workshop is “Integrating Machine Learning into Geographic Research”. It will introduce students to the fundamental concepts and techniques related to machine learning, and how to integrate machine learning into geographic research. This workshop is designed to help students overcome some of the challenges posted by the pandemic by leveraging a free cloud computing platform, Google Colab, that allows students to build machine learning models on super computers safely at home and for free. The main programming language for this workshop will be Python, and the main machine learning package will be scikit-learn. Students can find more details or register here: https://www.eventbrite.com/e/132423234459

Workshop schedule

The workshop will start on Feb 8 and end on Feb 12 (note: the interactive sessions will not be recorded.)

• Monday, asynchronous: Intro to GeoAI and Google Colab: A pre-recorded lecture consisting of two videos is made available. We will give a brief introduction to geospatial artificial intelligence (GeoAI) and help students set up the working environment on Google Colab.
• Tuesday, synchronous, 9:00 – 11:00 AM (ET): Python Refresh and GeoPandas: We will have a 2-hour live session on Zoom. This session will help you refresh Python programming basics and learn how to work with shapefile data using the GeoPandas library.
• Wednesday, synchronous, 9:00 – 11:00 AM (ET): Machine learning for Geography: We will have a 2-hour live session on Zoom. This session will cover the basics of preparing geographic data for machine learning models and implementing a model (Random Forest) using the scikit-learning library.
• Thursday, asynchronous: Exercise: Building Your Own Machine Learning Model A simple exercise will be released at the end of the session on Wednesday. Students are expected to work on this exercise on Thursday, in which you will be asked to build your own model using scikit-learning.
• Friday, synchronous, 9:00 – 11:00 AM (ET): Recap and the Road Forward: We will have a 2-hour live session on Zoom. In this session, I will review the exercise that you have worked on Thursday, and answer any questions. I will also share some resources for studying machine learning and AI beyond this workshop.

This workshop series is supported by AAG staff Coline Dony and Julaiti Nilupaer. Our GeoAI Lab thanks their great effort and dedication to help our graduate students go through this challenging time!

The members of our GeoAI Lab wish you Merry Christmas and a happy new year! 2020 has been challenging and difficult. Let’s hope for a better 2021!

Where is Santa? Let’s find him: https://santatracker.google.com/

Yingjie recently had an interview with the Britannica Kids Encyclopedia to discuss geospatial technologies and AI. Hope more from the younger generations will be interested in joining the geospatial world!

Interview video on YouTube:

Our new paper on “Aligning geographic entities from historical maps for building knowledge graphs” is accepted in the International Journal of Geographical Information Science.

Historical maps contain rich geographic information about the past of a region. They are sometimes the only source of information before the availability of digital maps. Despite their valuable content, it is often challenging to access and use the information in historical maps, due to their forms of paper-based maps or scanned images. It is even more time-consuming and labor-intensive to conduct an analysis that requires a synthesis of the information from multiple historical maps. To facilitate the use of the geographic information contained in historical maps, one way is to build a geographic knowledge graph (GKG) from them. This paper proposes a general workflow for completing one important step of building such a GKG, namely aligning the same geographic entities from different maps. We present this workflow and the related methods for implementation, and systematically evaluate their performances using two different datasets of historical maps. The evaluation results show that machine learning and deep learning models for matching place names are sensitive to the thresholds learned from the training data, and a combination of measures based on string similarity, spatial distance, and approximate topological relation achieves the best performance with an average F-score of 0.89.

For more information, please refer to our full paper:
Kai Sun, Yingjie Hu, Jia Song, and Yunqiang Zhu (2020): Aligning geographic entities from historical maps for building knowledge graphs. International Journal of Geographical Information Science, in press.