An edge device that worked fine in the lab and starts misbehaving after a few weeks in the field is a classic. It slows down, drops messages, reboots at odd hours, or goes silent until someone power-cycles it. The cause is nearly always one of four resources running out: memory, storage, CPU time, or thermal headroom. The skill is working out which, without a keyboard and monitor attached.

First, Collect Evidence Remotely

Resist the urge to guess. Before changing anything, get data from the device itself. If your firmware or agent does not report basic resource metrics, that gap is your first finding.

At minimum you want, over time: memory in use and available, storage free space, CPU load, temperature, uptime, and the reason for the last reset. A single snapshot is rarely enough. The shape over days is what tells the story. A line that rises steadily until the device resets points somewhere very different than a spike that happens at the same hour each day.

If you cannot connect at all, collect what you can from the cloud side: when did the messages stop, and was there a pattern in the preceding reconnects or message sizes?

Symptom One: Gradual Slowdown Ending in a Reboot

This pattern, with memory use climbing across days, strongly suggests a leak. Something allocates and never releases: buffers for messages that never get acknowledged, connection objects that are not closed after a failed reconnect, or an unbounded in-memory queue.

Look at what grows. If your application buffers data during connectivity loss, check whether that buffer has a cap. A common trap is a queue that works perfectly while the network is up and consumes every byte of RAM during a long outage, which is exactly when nobody is watching.

Fixes: put a hard limit on every queue and decide what gets dropped when it is full, close resources on every error path, and add a watchdog that restarts the process cleanly before the operating system does it badly.

Symptom Two: Failures After Weeks, With Write Errors

If the device runs but cannot save data, check storage. Logs are the usual culprit: debug logging left on, or logs that never rotate, quietly filling the disk. Local databases and cached files that are never pruned do the same.

Flash storage adds a second problem: it wears out with repeated writes. A process that writes frequently to the same area, such as a once-per-second state file, can shorten the life of the storage. Check the hardware documentation for endurance guidance, reduce write frequency, batch writes, or keep high-churn data in memory and persist it less often.

Fixes: rotate and cap logs, set explicit retention on cached data, and reserve free space so the system has room to operate.

Symptom Three: Sluggish at Particular Times

If problems cluster at certain times of day or after specific events, look at CPU. Common culprits are a processing job scheduled on all devices at the same minute, retry storms after a connection returns, and inference or compression work that was sized for typical load rather than peak.

Retry behavior deserves particular suspicion. When many messages are queued during an outage and the link comes back, naive code sends everything at once, saturating the CPU and the uplink. Pace replay, and add jitter to retry timers so a whole fleet does not synchronise.

Symptom Four: Problems on Hot Days or in Enclosures

Devices in sealed boxes, sunlit cabinets or vehicles often hit thermal limits. Processors reduce their clock speed to protect themselves, so performance falls exactly when the environment is hottest, and sustained heat can shorten component life or trigger resets.

Correlate temperature readings with performance dips. If the pattern lines up, the fix may be physical: ventilation, a heat sink, shade, or a different enclosure. It can also be software: reduce the duty cycle of heavy tasks, defer them to cooler periods, or lower processing load.

Diagnosis at a Glance

Symptom	Likely resource	First check	Typical remedy
Slows over days, then resets	Memory	Memory trend over time	Cap queues, fix leaks, watchdog
Write errors, cannot save	Storage	Free space and log growth	Rotate logs, prune caches
Lag at fixed times or after outages	CPU	Load versus schedule and retries	Stagger jobs, add jitter
Worse on hot days	Thermal	Temperature against performance	Cooling, lighter duty cycle
Random resets with no trend	Power	Reset reason and supply voltage	Power supply review

The last row is easy to overlook. Brown-outs from marginal power supplies cause resets that look like software crashes. If reset reasons point to power, investigate hardware before rewriting code.

A Hypothetical Example

Suppose a fleet of edge gateways in roadside cabinets begins rebooting every few days. Metrics show memory climbing steadily on affected units, but only in sites with poor cellular coverage. The team finds that the message buffer for the cloud uplink has no limit: when the network is bad, messages pile up in RAM.

They cap the buffer, write older messages to a bounded file on disk with the oldest dropped first, and pace the replay when the link returns. Reboots stop. The bug was invisible in the office because the office network never failed.

Preventing the Next One

Test the failure modes, not only the happy path. Run a soak test for days with simulated network loss, full disks and high temperature. Add resource metrics to the heartbeat so you see trends before they become incidents. And design a safe degradation: when resources get tight, shed the least important work first rather than letting the operating system choose.

Frequently Asked Questions

How do I diagnose a device I cannot log into? Use the data it already sends: reset reasons, uptime, and any resource metrics in its heartbeat. If these are missing, add them to the next firmware release.

Is a scheduled daily reboot a good fix? It can be a stopgap that hides a leak, and it can mask the real cause. Use it as a safety net while you fix the underlying problem, not as the solution.

How much free storage should I keep? Enough that logging, updates and temporary files always have room. Size it from your largest update package plus working space, and check hardware documentation for flash guidance.

Could the problem be the network and not the device? Yes, but network trouble often triggers device trouble through buffering and retries. Examine both sides together.

When should I replace the hardware? When the workload genuinely exceeds the hardware’s limits after reasonable optimisation, or when thermal and power margins cannot be fixed in software.

Next Step

Pick one failing or suspicious device class and add four numbers to its heartbeat: free memory, free storage, temperature and reset reason. Collect two weeks of data before deciding what to fix.