Internet of Things (IoT) devices have become ubiquitous and spread across many application domains including the industry, transportation, healthcare, and households. However, the proliferation of the IoT devices has raised the concerns about their security - many manufacturers focus only on the core functionality of their products due to short time to market and low cost pressures, while neglecting security aspects. Moreover, there is no established or standardized method for measuring and ensuring the security of IoT devices. Consequently, vulnerabilities are left untreated, allowing attackers to exploit IoT devices for various purposes, such as compromising privacy, recruiting devices into a botnet, or misusing devices to perform cryptocurrency mining. In this paper, we present a practical Host-based Anomaly DEtection System for IoT (HADES-IoT) as a novel last line of defense. HADES-IoT has proactive detection capabilities, provides tamper-proof resistance, and can be deployed on a wide range of Linux-based IoT devices. The main advantage of HADES-IoT is its low performance overhead, which makes it suitable for the IoT domain, where state-of-the-art approaches cannot be applied due to their high-performance demands. We deployed HADES-IoT on seven IoT devices and demonstrated 100% effectiveness in the detection of current IoT malware such as VPNFilter and IoTReaper; while on average, requiring only 5.5% of available memory and causing only a low CPU load.