In this post, I will demonstrate loading my criminal activity data into ElasticSearch sot it can be explored, analyzed and visualized in Kibana. For instructions on installing and configuring the Elastic (formerly ELK) Stack, see my previous post. Although this post will specially reference the crime data from my PostgreSQL database, I will include additional…
In this post, I will cover getting open source COVID-19 data for the United States using Julia. The data pipeline demonstrated here is very simple example and could easily be adapted into a Prefect, Apache NiFi or Apache AirFlow ETL process. Data Search Performing a quick search on DuckDuckGo I got The COVID Tracking Project,…
In this post, I will cover getting open source COVID-19 data for the United States using Python. The data pipeline demonstrated here is very simple example and could easily be adapted into a Prefect, Apache NiFi or Apache AirFlow ETL process. Data Search Performing a quick search on DuckDuckGo I got The COVID Tracking Project,…
In this post, I will cover getting open source COVID-19 data for the United States using R. The data pipeline demonstrated here is very simple example and could easily be adapted into a Prefect, Apache NiFi or Apache AirFlow ETL process. Data Search Performing a quick search on DuckDuckGo I got The COVID Tracking Project,…
Exploring Mapping Data In a continuation from part 2a, this post will explore the spatial points datasets from my database. I need to assess what each dataset contains. To aid in the exploration of spatial data, I will demonstrate plotting spatial points. This can be a lot easier to look at this particular structure in…
Exploring Mapping Data My next exploration task is the mapping/geospatial tables in my database. I need to assess the what each dataset contains. To aid in the exploration of spatial data, I will demonstrate plotting of spatial points and polygons. This can be a lot easier to looking at this particular structure in tabular or…
Exploring Criminal Activity Data My primary data set to explore is the crime table in my database. I need to assess the values and consistency across each of the fields available in what was provided by the source. It is possible for data to be a little inconsistent across each of the years over the…
Thus far I have gathered and stored my data for the vast majority of the project into my PostgreSQL database. Now its time to explore the data. Over the next series of posts for this project, I plan to share different methods of exploring the data. Initially I’ll demonstrate using R to tap into my…
I have gathered three quarters of my branches on the “Other Data Sources” category. The next up is to get real estate data based on the project plan. Real Estate Data Finding good real estate data took a some work to explore and gather. I’m sure there is a lot of data available privately for…
In this post I will go over setting up a geospatial database using PostGIS, an extension to PostgreSQL. For information about setting up a PostgreSQL database please refer back to my previous Data Storage. Below is the project plan to ensure I load up all my downloaded map data. Installing and Setting up First you…
