# Assignment 3


## Task 1: Working with Mongo - Advanced


## Areas covered:
- Working with NoSQL
- Working with Pandas


## Description:

You are given with a dataset of a restaurant review in the form of a JSON file. The end goal of the project is to create an API interface that will provide the following: 
1. Business name with maximum number of highest average review.
2. Which cuisine has the highest number of restaurants?
3. Generate Excel Report based on Cuisine, Name and borough
Sample Document:
```json
{
  "address": {
    "building": "120",
    "coord": [
      -73.9998042,
      40.7251256
    ],
    "street": "Prince Street",
    "zipcode": "10012"
  },
  "borough": "Manhattan",
  "cuisine": "Bakery",
  "grades": [
    {
      "date": {
        "$date": "2014-10-17T00:00:00.000Z"
      },
      "grade": "A",
      "score": 11
    },
    {
      "date": {
        "$date": "2013-09-18T00:00:00.000Z"
      },
      "grade": "A",
      "score": 13
    },
    {
      "date": {
        "$date": "2013-04-30T00:00:00.000Z"
      },
      "grade": "A",
      "score": 7
    },
    {
      "date": {
        "$date": "2012-04-20T00:00:00.000Z"
      },
      "grade": "A",
      "score": 7
    },
    {
      "date": {
        "$date": "2011-12-19T00:00:00.000Z"
      },
      "grade": "A",
      "score": 3
    }
  ],
  "name": "Olive'S",
  "restaurant_id": "40363151"
}
```

Bonus Points: Use Mongo Aggregate framework
### Tools to use: 
1. Pycharm / VSCode
2. Robo3T / Studio3T / MongoDB Compass
3. PyMongo

### Reference:
https://www.mongodb.com/docs/manual/tutorial/query-documents/
https://www.mongodb.com/docs/manual/reference/operator/aggregation-pipeline/
https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_excel.html
https://fastapi.tiangolo.com/advanced/custom-response/#fileresponse
https://pymongo.readthedocs.io/en/stable/



## Task 2: Working with Timeseries

You are given with a dataset of weather in the form of a CSV file. The end goal of the project is to create an API interface that will provide the following:

1. Get daily, monthly, weekly and monthly aggregate (min, max, and average) of the data and generate report in Excel format.

Sample Document:

|Datetime        |AEP_MW|
|----------------|------|
|31/12/2004 01:00|13478 |
|31/12/2004 02:00|12865 |

### Tools to use: 
1. Pycharm / VSCode
2. Pandas 

### Reference:
https://kairosdb.github.io/docs/restapi/QueryMetrics.html

https://pypi.org/project/kairosdb-python/


## Task 3: Working with MQTT & REDIS

# Description
Data from different sites will be pushed with frequency of 10 seconds for the parameters PM10,PM2.5,SO2,NO2 via mqtt.

data can be of different quality - Good ( 0 ), Maintainance ( 1 ), Error ( 2 )

Based on the quality of data update to different redis database.

sample data format: 
```json
  {
		"data" : { "PM10" : 100 , "PM2.5": 23, "SO2":21, "NO2": 32}
		"site_id" : "site_100",
		"data_quality": 1
		}
```
	   
Use Redis for caching/storing information

create consumer's which consumes data from these topics and store to a redis db based on data quality.

### Tools to use: 
1. Pycharm / VSCode
2. MQTT - (paho-mqtt)
3. REDIS - (redis)

### Reference:
https://www.emqx.com/en/blog/how-to-use-mqtt-in-python

https://docs.redis.com/latest/rs/references/client_references/client_python/

