Scary Cabin

I’ve been thinking about the data collection I do. Specifically about what machines are doing the collection and what type of system load and stability my current system has.

Today, I have two primary data collection functions running.

1) Home electricity production/usage data.
2) Outside weather data.

For the electricity data I have a daemon running on my main machine that pulls the raw data from a monitor attached to the mains. The raw data is processed and stored in a MySQL database. The monitor is attached to the machine via a USB port.

For the weather data I have a program running on a Raspberry PI that intercepts raw data from an Acurite 5-n-1 sensor bridge, processes it and stores it in a MySQL database.

The MySQL database resides on a dedicated disk partition and the server is running on my main machine. The database is also used for a few other things like a bug tracking database. The clients of the database are my web server (also running on the main machine) and the home automation software package (currently running in a vmware Windows instance on the main machine).

What if this was all consolidated on to a dedicated home data collection server? That’s the question I’m asking myself right now. That leads to a few more questions.

- What exactly should be consolidated? Data collection daemons, MySQL database, webserver?
- Can a Raspberry PI handle all of this?
- Would it improve stability?
- What hardware is needed in addition to the Raspberry PI?
- Can an SSD drive be used with the database?
- How easy will be to back this all up?

After a bit more thought and some research, I think I have answers for most of the questions. All three major components should be consolidated. Given how much interaction there is between the data collection daemons and the database it just makes sense to have them on the same machine if possible. Same for the web server and database. The disk space needed to support both database and web server is not excessive and easily handled by a low cost SSD. Currently it’s less than 50GB and it’s not growing that fast. A 120GB drive should suffice for a number of years. The Raspberry PI should handle the data collection daemons easily. It should also be able to handle the database. The web server handles mostly private needs so I think that’s fine, but will need to do some testing to be sure.

For hardware I think just the Raspberry PI with OS on an SD card. Probably a USB hub to handle the addition network interface, electricity monitor and SSD disk. And of course the SSD disk 120GB or 240GB.

The SSD drive can be shared via NFS and/or SMB and that provides both a way to back it up and add web content easily.

So time to start prototyping. I have an extra PI and spare hard drives so I should be able to install most of the software and make sure the services are running. Then it’s just a matter of switching over.

Now what other data could I collect using this setup?