atelle's Space

Projects

New project

2023.04.17 00:01:01 by atelle

I love HomeLabs, and run a number of services and systems both at home and in the cloud.

As I set up more and more virtual servers and various systems running on these virtual machines I started to have a problem: when things went down I didn't find out until much later.

There are a couple of solutions to this. One can set up UptimeRobot or Pingdom to monitor your systems or set up an instance of UptimeKuma, a self-hosted alternative to one of the solutions above.

All the solutions work well, but the notification solution is not always optimal. UptimeRobot and Pingdom offer to send out an email (free) or sms (unsure of price) when things go down or up.
With UptimeKuma you have many more options: Email, Alerta, WebHooks, Telegram, etc. What makes UptimeKuma a little more complicated is that you have to set up the notification system yourself, for example configuring SMTP to receive emails. But those who set something up themselves have no problems with this.

One of the problems I had with UptimeRobot and Pingdom was that I could not monitor internal systems. This means that everything I run behind my firewall is not monitored.
Therefore, I chose to use UptimeKuma as the primary monitoring system, and have set up 3 instances of this in 3 different locations (one at home, one in a data center and one in Azure)

Another problem I had was that I had no overview of all the alarms that came in. I have configured all three UptimeKuma instances to send me a Telegram notification when something critical goes down, but it is difficult to have control over this in the long run.

To solve this problem, I chose to build an alarm center from scratch. I have tried to use systems that already exist, but these are often made for companies, and cost a lot of money for licenses. It was also a pretty cool project, where I learned a lot.

Here are some pictures of what the system looks like. I chose to use Bootstrap 5.3 as my design framework because it looks elegant, is easy to use, and supports both light and dark modes.

(Click on the pictures for full size)

The system currently has 3 APIs:

Bash - Allows me to send my own messages directly to the case system from a linux system. This allows me to configure notifications on various services I run, for example if a cronjob fails, or if the server restarts.

WebHook - A simple system that receives notifications directly from UptimeKuma. This means that tickets are generated instantly, and can receive large amounts of alarms at once.

Email - This system started using Cloudmailin. With this system I could send an email to Cloudmailin, which was forwarded to my system via WebHook. This system worked quite well, but is dependent on Cloudmailin, and has limitations on how many API requests can be made per month. To avoid this, I chose to set up my own e-mail server. Here I can send an unlimited number of emails, without reaching any limit per month. In order to get the e-mails into the case system, I have written a small API that connects to the e-mail server with IMAP, and retrieves all e-mails that were sent from a specific gmail account (Since I use Gmail's SMTP to send out e-mails). The API also ensures that an email is only added once, and ignores this on the next query. It's not a completely perfect system, since the API runs through the crontab once a minute on a server. This means that it can take up to 60 seconds from when an email is received until it is imported into the case system.

The project website is https://ticketcore.net/.

About

My private space on the web

Domain Portfolio

norhood.com (Registered 2012-06-03)

fawdaw.com (Registered 2014-06-21)

pxcd.org (Registered 2018-11-28)

akseltelle.com (Registered 2021-07-13)

atelle.xyz (Registered 2021-12-18)

atelle.org (Registered 2022-10-19)

buildnorge.no (Registered 2022-10-20)

hobbyrack.no (Registered 2022-10-28)

hobbyserver.no (Registered 2022-10-28)

hellofriend.no (Registered 2022-10-31)

telle.at (Registered 2022-11-02)

telle.pw (Registered 2023-01-02)

ticketcore.net (Registered 2023-04-24)

monitorcore.net (Registered 2023-06-30)

crypt3r.com (Registered 2023-12-10)

https.no (Registered 2024-03-17)

bilreg.net (Registered 2024-04-02)

atelle's Space

Welcome to atelle's Space!

New project

About

Domain Portfolio

Archive