9.7  Typical Edge-AI Hardware

Peak operations per second are not application performance. Measure the complete pipeline on the target hardware: acquisition, resizing, preprocessing, inference, postprocessing, communication, and actuation. Thermal throttling can make a five-minute benchmark misleading for a system that must operate all day.

9.8  A Practical Inference Pipeline

A vision system commonly performs:

1. image acquisition and timestamping;
2. quality checks for exposure, blur, obstruction, and missing frames;
3. resizing, color conversion, normalization, or cropping;
4. model inference;
5. postprocessing such as thresholding and non-maximum suppression;
6. tracking or temporal filtering;
7. decision policy; and
8. logging, visualization, or bounded actuation.

For a mobile agricultural robot, a detector might locate crop rows, weeds, people, or obstacles. A model output is not yet a steering command. Geometry, temporal consistency, confidence, vehicle state, speed limits, and an independently enforced safe stop must be applied before motion.

9.9  Understanding Model Tasks

Classification: Assign a label to an input, such as healthy or diseased leaf.
Object detection: Estimate object classes and bounding boxes, such as people and vehicles in an image.
Segmentation: Assign a class to pixels, useful for crop-row, traversable-area, or weed mapping.
Regression: Estimate a continuous value, such as biomass or remaining battery life.
Anomaly detection: Estimate whether a sample differs from learned normal behavior.
Tracking: Associate detections across time and estimate motion.
Forecasting: Predict future values from a time series.

The model type should match the engineering question. A classifier applied to an entire field image cannot localize the weed that a robot must remove. An anomaly score may indicate unusual vibration without explaining the mechanical cause.

9.10  Metrics That Describe Reality

For binary classification, let TP, TN, FP, and FN denote true positives, true negatives, false positives, and false negatives:

These ratios require nonzero denominators. If a class has no actual or predicted positives, report the affected metric as undefined or state the evaluation library’s explicit convention. Accuracy is misleading when classes are imbalanced. If only one image in 100 contains a person, a model that always predicts “no person” obtains 99% accuracy and is operationally useless. Precision answers “when the model reports positive, how often is it right?” Recall answers “of the actual positives, how many did it find?”

Threshold selection is a cost decision. A false plant-treatment command wastes chemical or energy; a missed human obstacle can be catastrophic. Report performance over relevant thresholds and define the operating point from consequences rather than convenience.

For regression, useful measures include mean absolute error

and root mean square error

These aggregate errors must be complemented by residual plots and performance across seasons, ranges, locations, devices, and operating conditions.

For object detection and segmentation, intersection over union, average precision, per-class recall, small-object performance, and inference latency are relevant. For an embedded system also report:

• median and high-percentile latency, not only the average;
• throughput in frames or samples per second;
• missed deadlines and dropped frames;
• peak RAM and accelerator memory;
• energy per inference or average power;
• sustained temperature and clock behavior; and
• end-to-end decision latency.

9.11  Data Splitting and Leakage

Training data fits the model, validation data supports choices, and test data estimates final generalization. The test set must not influence tuning. Data leakage occurs when information unavailable during real deployment enters model development or evaluation.

Randomly splitting individual video frames is often invalid because adjacent frames are nearly identical. Agricultural images from the same plant, plot, flight, day, or camera may also be strongly correlated. Split by an independent unit such as field, farm, animal, robot run, date, or season according to the intended claim.

9.12  Dataset and Model Lifecycle

A reproducible model release should record:

• dataset identity, collection conditions, consent or lawful basis, and provenance;
• inclusion and exclusion criteria;
• annotation instructions and reviewer agreement;
• train/validation/test split method;
• preprocessing and augmentation;
• architecture, initialization, optimizer, and training settings;
• software dependencies and random seeds where meaningful;
• output semantics and decision threshold;
• target hardware and measured resource use;
• known limitations and excluded operating conditions; and
• model, configuration, and firmware versions.

Use a model registry or equivalent inventory. Rollout should support staged deployment, monitoring, and rollback. A hash can confirm that the loaded artifact matches the approved artifact. Signing can authenticate its source.

9.13  Drift and Out-of-Distribution Inputs

Data drift is a change in the input distribution. Concept drift is a change in the relationship between inputs and desired outputs. Causes include seasons, lighting, camera replacement, dirt, sensor aging, new crop varieties, different operators, and changes in process behavior.

A model can be confidently wrong on input unlike its training data. Softmax confidence is not a guarantee of correctness. Practical defenses include input-quality checks, operating-envelope monitoring, conservative thresholds, ensembles or uncertainty estimates where justified, periodic labeled audits, and fallback when the input is outside the validated domain.

9.14  Compression and Deployment Formats

Quantization represents weights and sometimes activations with reduced numerical precision. Pruning removes selected weights or structures. Knowledge distillation trains a smaller student model to reproduce useful behavior of a larger teacher. These techniques may reduce memory, latency, or energy, but can affect classes and conditions unequally.

Common deployment representations include TensorFlow Lite, ONNX, OpenVINO formats, and vendor-specific optimized engines such as TensorRT. Conversion can change supported operators, numerical behavior, dynamic shapes, and preprocessing expectations. Validate the converted model on the same locked test set and on the actual target.

This benchmark expects an indexable collection and a callable predict. It excludes acquisition and postprocessing unless they are deliberately placed inside predict. For an asynchronous accelerator, supply a synchronization callback that waits for the relevant device work to finish; otherwise the timer may measure only submission time. For example, PyTorch provides torch.cuda.synchronize for CUDA work. The nearest-rank p95 is coarse for a small sample; report the count and run long enough to observe thermal effects. Record the target configuration and measure throughput separately when pipelining matters.

9.15  Useful Embedded-AI Examples

The right baseline may be a threshold, linear regression, physical model, or conventional image-processing method. AI is justified only when it produces a measurable benefit under representative conditions.

9.16  Local Large Language Models

A large language model predicts token sequences from context learned during training. It does not retrieve truth from a guaranteed database and does not inherently understand the physical system. A local LLM executes on equipment controlled by the organization or user rather than sending every request to an external inference service.

Potential benefits include local operation, reduced external disclosure, predictable local integration, and the ability to use private documents. Limitations include substantial memory and compute requirements, slower inference on small hardware, model maintenance, licensing considerations, and continued risk of fabricated or inappropriate output.

Important terms are:

9.17  Selecting and Running a Local LLM

Model choice depends on task quality, license, language, context length, memory, latency, energy, and supported runtime. Parameter count alone does not determine usefulness. Quantization can make a model fit on smaller equipment, but quality and speed must be tested on the actual prompts and hardware.

A local runtime can expose an HTTP API to applications. This is convenient but creates a network service that must be authenticated or bound to a trusted interface. Do not assume that “local” automatically means private: prompts may be logged, stored in shell history, included in backups, exposed through dashboards, or sent through external tools.

Requiring JSON improves parsing, not truthfulness. Validate schema, values, authorization, data freshness, and physical constraints after generation. A model’s self-reported confidence is not calibrated probability and must not grant authority. The caller must reject parsing or validation errors, cap the response before buffering it, and separately enforce tool permissions and rate limits.[74]

9.18  Retrieval-Augmented Generation for Technical Notes

Retrieval-augmented generation can support a local maintenance assistant:

1. ingest approved manuals, wiring diagrams, and procedures;
2. divide them into traceable chunks;
3. create a searchable index;
4. retrieve passages relevant to a question;
5. ask the model to answer only from those passages; and
6. display document identity, revision, and citations to the operator.

This reduces but does not eliminate fabrication. Retrieval may select an obsolete manual, omit an exception, or find a procedure for another equipment variant. High-consequence instructions require authoritative document control and human verification.

9.19  Prompt Injection and Tool Safety

Prompt injection occurs when untrusted content attempts to redirect the model or override intended behavior. A webpage, log entry, email, QR code, or retrieved document can contain hostile instructions. The model should never determine its own authority.

For an LLM-enabled embedded or automation system:

• separate data from instructions where possible;
• maintain a small allowlist of narrow tools;
• validate every tool argument independently;
• require confirmation for consequential operations;
• run tools with least privilege and bounded time/resources;
• prevent arbitrary shell, file, database, or network access;
• log proposed and executed actions with secret redaction; and
• assume external and retrieved content may be malicious.

9.21  Fallback and Graceful Degradation

Define behavior for each of the following:

• the model service is unavailable;
• inference exceeds its deadline;
• input quality is insufficient;
• model confidence or uncertainty is unacceptable;
• the input is outside the validated operating domain;
• the accelerator overheats or runs out of memory;
• a new model fails after deployment; and
• the network, broker, dashboard, or database is unavailable.

Possible degraded behaviors include retaining the last safe command only for a bounded lease, returning to a conservative rule-based strategy, reducing speed, suspending automatic action, storing data locally, and requesting human intervention. A fallback must itself be tested. “The operator will notice” is not a dependable mechanism unless the detection, notification, response time, and authority have been designed.

9.22  Privacy and Data Governance

Local inference can reduce external data transfer but does not solve privacy automatically. Cameras, audio, location, identifiers, and behavioral records may still be personal or sensitive data. Apply data minimization: collect only what the stated purpose requires, retain it only as long as justified, protect access, and document who can use it.

Separate raw data, annotations, derived features, model outputs, and audit logs. Each may have different retention and access requirements. Before using captured data for a new research or teaching purpose, verify that the original basis and participant information permit it. Anonymization is difficult when video, trajectories, or location patterns can identify individuals indirectly.

9.23  Security of Home Assistant and AI Services

An integration platform concentrates credentials and control. Protect it accordingly:

• keep Home Assistant, add-ons, operating system, and integrations updated;
• use strong unique authentication and multifactor authentication where supported;
• do not expose services directly to the Internet without a justified secure architecture;
• segment IoT devices from administrative systems;
• use TLS and narrow MQTT access-control lists;
• store secrets outside public repositories and rotate them after exposure;
• disable unused integrations and remove abandoned accounts;
• back up configuration and test restoration;
• restrict local model APIs and tool permissions; and
• monitor authentication failures, unexpected entity creation, and unusual commands.

Availability is part of security. A malicious or faulty client can flood MQTT, exhaust model memory, fill storage with images, or generate repeated automations. Apply quotas, payload limits, timeouts, queue bounds, and resource isolation.

9.24  Practical System Example

Consider a greenhouse monitoring system with BME280 environment sensing, soil-moisture probes, a camera, ventilation fans, and irrigation valves.

1. An ESP32 validates sensor readings and publishes time-stamped MQTT measurements.
2. Home Assistant presents current values, data age, device availability, and manual controls.
3. A deterministic automation alerts on prolonged temperature excess and may request ventilation within maximum-runtime constraints.
4. A Raspberry Pi or Jetson performs local leaf-image inference and publishes a disease-risk observation with model version and image-quality status.
5. The AI result creates an inspection request; it does not directly apply treatment.
6. A local LLM summarizes recent alarms using approved records and cites their timestamps.
7. A PLC or microcontroller enforces pump interlocks, dry-run protection, maximum valve duration, and emergency behavior.
8. Operators can override ordinary automation, while physical safety protection remains independent.

This architecture uses each tool where it is strong: microcontrollers for bounded local behavior, Home Assistant for integration and supervision, edge AI for perception, an LLM for human-facing synthesis, and deterministic hardware/software for authority.

9.25  Guided Laboratory

9.26  Self-Assessment and Job-Interview Questions

The following questions assess this course material while preparing students for embedded-systems, IoT, automation, robotics, and edge-AI interviews. Strong answers state assumptions, identify failure modes, and connect software behavior to physical consequences.

1. What is the difference between a Home Assistant device and an entity?
2. Explain triggers, conditions, and actions using an agricultural example.
3. Why does hysteresis matter in physical automation?
4. What is the difference between requested, accepted, and measured actuator state?

5. Why can a retained MQTT message be dangerous on a command topic?
6. Compare monitoring, automation, feedback control, and a safety function.
7. Define edge AI and give three reasons to use it.
8. Why is model accuracy alone inadequate?
9. A dataset contains 1,000 images, of which 20 contain a hazard. A model reports every image as safe. What is its accuracy, and why is this result unacceptable?
10. Distinguish precision and recall. Which would you prioritize for person detection on a moving robot?
11. What is data leakage, and why can a random video-frame split often invalid?
12. Distinguish data drift and concept drift.
13. Why must a converted or quantized model be tested again?
14. What measurements belong in an edge-inference benchmark?
15. What is a local LLM, and what does “local” fail to guarantee?
16. Why is low temperature in an LLM not a truth guarantee?
17. What is retrieval-augmented generation, and what can still go wrong?
18. What is prompt injection?
19. Why is valid JSON from an LLM not necessarily a valid engineering command?
20. Design a safe authority boundary for an AI-assisted irrigation system.
21. What should happen when an edge model misses its deadline?
22. Why should a rule-based baseline be implemented before claiming an AI benefit?
23. How would you detect that a Home Assistant sensor is stale rather than merely unchanged?
24. What evidence would persuade you that an automation survives a broker outage?
25. When should Home Assistant not be used to control a process?

9.27  Answers and Discussion

1. A device represents a physical or logical product; an entity represents one observable or controllable function belonging to that product. One weather node may contain temperature, humidity, battery, and availability entities.
2. A soil-moisture value remaining below a threshold for 30 minutes can be the trigger. Conditions may require automatic mode, adequate tank level, and no pump fault. Actions may request a bounded irrigation cycle and notify the operator. Physical protection remains local.
3. Noise near one threshold can repeatedly switch an actuator. Different on and off thresholds create a dead band, reducing chatter, wear, and unstable interaction. Persistence and minimum dwell time may also be required.
4. Requested state is what a client asks for; accepted state means the controller validated and admitted the request; measured state is obtained from feedback about the physical output. They can differ because of rejection, delay, disconnection, or mechanical failure.
5. A broker can deliver a retained command to a device after reconnection or restart. The command may be old but still syntactically valid. Prefer non-retained commands with identifiers, expiration, authorization, and explicit acknowledgement.
6. Monitoring observes; automation executes explicit event-driven rules; feedback control continuously acts on process error; a safety function prevents or mitigates unacceptable risk with an assurance level appropriate to that risk. One platform can support several roles, but the roles must not be confused.
7. Edge AI performs inference near the source. Reasons include low latency, offline operation, bandwidth reduction, privacy, and local filtering. These benefits must be balanced against compute, power, thermal, and maintenance limits.
8. Accuracy hides class imbalance and different error costs. The engineer must consider confusion matrices, precision, recall, class-specific performance, thresholds, latency, resource use, robustness, and representative field conditions.

9. The accuracy is 980/1000 = 98%. Recall for the hazard class is zero because all 20 hazards are missed. A high aggregate accuracy therefore conceals complete failure at the purpose of the detector.
10. Precision measures how often positive predictions are correct; recall measures how many actual positives are detected. Person-safety perception usually demands high recall, but false positives and stopping behavior also require system-level analysis. The detector must not be the only protective mechanism.
11. Leakage allows information correlated with the answer or test cases into training or tuning. Adjacent video frames are near duplicates, so random frame splitting lets the model see almost the same scenes in both training and test sets. Split by independent runs, locations, subjects, or dates.
12. Data drift changes the input distribution, such as lighting or camera properties. Concept drift changes the mapping from input to desired output, such as a changed relationship between symptoms and diagnosis. Both can reduce deployed performance.
13. Conversion may change precision, operators, preprocessing, shapes, or numerical results. Performance and task quality must be remeasured on the locked evaluation set and target hardware.
14. Report end-to-end median, high-percentile, and maximum latency; throughput; missed deadlines; memory; power or energy; sustained temperature and clocks; model and runtime versions; preprocessing; batch size; and task accuracy on representative inputs.
15. A local LLM runs on locally controlled infrastructure. It does not guarantee secrecy, factual correctness, legal compliance, absence of logs, or offline behavior if the surrounding application calls external services.
16. Temperature changes sampling variability. A deterministic or low-variability answer can still repeat the same false claim. Factual confidence requires evidence and verification, not a generation setting.
17. Retrieval-augmented generation retrieves relevant material and supplies it to the model. Retrieval can miss the right passage, select obsolete or malicious text, confuse equipment variants, or be summarized incorrectly. Display authoritative sources and revisions.
18. Prompt injection is untrusted content designed to alter model behavior or induce unintended tool use. Retrieved pages, documents, sensor labels, and log messages must be treated as data rather than trusted instructions.
19. JSON proves only that parsing may succeed. The content can be false, stale, unauthorized, outside physical limits, or malicious. Deterministic validation and authorization remain mandatory.
20. Let AI estimate need and provide a recommendation. A deterministic policy checks sensor quality, data age, tank state, weather constraints, allowed duration, rate limits, and operator mode. A local controller applies valve and pump interlocks, and independent protection prevents dry running or excessive pressure.
21. Discard or mark the late result according to its event time, enter a defined degraded mode, expose diagnostics, and use a validated baseline or safe state. Acting on an old inference may be worse than having no inference.
22. A baseline reveals whether AI adds measurable value over simple logic and provides a fallback. Without it, complexity may be mistaken for improvement.
23. Carry an observation timestamp or monotonically increasing sequence from the source and calculate age independently. An unchanged valid temperature may be fresh; a repeatedly displayed retained value may be stale.
24. Demonstrate bounded local buffering or a stated discard policy, visible unavailability, no unsafe retained commands, controlled reconnection with backoff, duplicate handling, time-stamped replay, and recovery measurements after restoring the broker.

25. Do not use it as the sole controller where hard real-time timing, certified functional safety, deterministic machine sequencing, or independent emergency response is required. It remains useful as a supervisory and integration layer above an appropriate local controller.

9.28  Authoritative References and Verification Links

The platforms and interfaces in this field evolve. Students should verify versions and configuration using primary documentation:

• Home Assistant documentation
• Home Assistant automation documentation
• Home Assistant MQTT integration
• Home Assistant MQTT discovery
• Home Assistant security guidance
• MQTT project and protocol information
• TensorFlow Lite documentation
• ONNX documentation
• OpenVINO documentation
• NVIDIA TensorRT documentation
• NVIDIA Jetson documentation
• Ollama documentation for local model serving
• OWASP guidance for LLM application security