A simplified boot sequence is:

1. processor-internal boot code selects a boot source;
2. platform firmware initializes essential hardware;
3. firmware or a bootloader loads the kernel, device tree, and optional initial RAM file system;
4. the kernel initializes memory, scheduling, drivers, and file systems;
5. the root file system becomes available;
6. the kernel starts process ID 1, commonly systemd; and
7. dependencies, services, login facilities, and the application are started.

Embedded storage experiences repeated logs, database updates, package operations, temporary files, and application state changes. Flash storage wears at erase-block level and depends on a controller for wear leveling. A reported file write may still reside in caches. Sudden power removal can therefore lose recent data or leave application and file-system state inconsistent.

Common mitigations include:

• a regulated supply with adequate transient current and undervoltage monitoring;
• controlled shutdown and sufficient hold-up energy where required;
• industrial or high-endurance storage selected for the write workload;
• a read-only root file system with a writable data area or overlay;
• journaling, though journaling does not guarantee application-level transaction correctness;
• bounded logging and rotation rather than uncontrolled file growth;
• atomic visibility: write and validate a temporary file on the destination file system, then rename it;
• application transactions, checksums, sequence numbers, and recovery tests; and
• separation of replaceable software, configuration, durable data, and disposable cache.

For durable file replacement on Linux, flush application buffers, synchronize the temporary file with fsync(), rename it on the same file system, and synchronize the containing directory. Check and handle errors at each step; atomic name replacement alone does not make data durable [6]. The sync command synchronizes buffered file-system writes, but cannot repair an inadequate power architecture or guarantee behavior of every storage controller. Reliability claims must be established through repeated fault testing, not assumed from the file-system name.

Provisioning converts a generic operating-system image into an identifiable, secure, and maintainable node. Record the image source and checksum, distribution release, architecture, hostname, account policy, network configuration, enabled services, package changes, application revision, and recovery method.

An initial checklist should include:

1. verify image authenticity and write it using a trusted tool;
2. boot with a known power supply and inspect undervoltage or thermal warnings;
3. change default credentials and remove unused accounts;
4. set a unique hostname and document how the address is assigned;
5. update deliberately and record the resulting software versions;
6. enable only required interfaces and services;
7. configure time synchronization and timezone policy;
8. establish key-based administrative access;
9. create the application identity and directories; and
10. capture a baseline before deploying project software.

Fleet devices require unique identity. Cloning an image can accidentally clone host keys, machine identifiers, credentials, and static addresses. A reusable image should generate or inject unique material during first boot or controlled enrollment.

SSH provides an encrypted channel for login, command execution, and file transfer. In public-key authentication, the client proves possession of a private key; the server stores the corresponding public key. The private key must remain protected. A passphrase reduces harm if the key file is stolen.

The server also proves its identity using a host key [18]. On first connection, the client has no automatic proof that the displayed fingerprint belongs to the intended node. Verify the fingerprint through a trusted channel. Blindly accepting changed host keys defeats protection against impersonation and may conceal that a device was reinstalled.

 1ssh-keygen -t ed25519 -C "eml-laptop"
 2ssh-copy-id student@eml-node.local
 3ssh student@eml-node.local
 4
 5# Inspect the server from an authorized account
 6sudo systemctl status ssh
 7sudo sshd -t
 8ss -lntp
 9journalctl -u ssh --since today

Harden remote access according to risk: restrict network exposure, prefer keys, disable unused accounts and authentication methods, apply least privilege, protect keys, keep software supported, and preserve auditable logs. Do not disable host-key checking merely to make automation convenient. Use managed known-host data instead.

Useful transfer methods include scp, sftp, and rsync over SSH. For repeatable deployment, verify the destination, preserve an application release identifier, and avoid overwriting the only known-good release.

A process is a running program. A daemon is a long-lived background process. A service is the useful function managed as an operational unit. Starting an application with python app.py & does not provide robust startup ordering, identity control, logging, restart limits, or boot integration.

systemd organizes work into units. A service unit declares how a process is started, under which identity, with which dependencies and restart policy. Unit files are configuration, so keep them in version control and validate them.

 1[Unit]
 2Description=EML environmental acquisition service
 3After=local-fs.target
 4StartLimitIntervalSec=120s
 5StartLimitBurst=5
 6
 7[Service]
 8Type=simple
 9User=eml-sensor
10Group=eml-sensor
11WorkingDirectory=/opt/eml/current
12EnvironmentFile=/etc/eml-sensor/environment
13ExecStart=/opt/eml/current/bin/acquire
14Restart=on-failure
15RestartSec=5s
16TimeoutStopSec=20s
17UMask=0027
18NoNewPrivileges=true
19PrivateTmp=true
20ProtectSystem=strict
21ProtectHome=true
22StateDirectory=eml-sensor
23StateDirectoryMode=0750
24LogsDirectory=eml-sensor
25LogsDirectoryMode=0750
26
27[Install]
28WantedBy=multi-user.target

 1sudo systemd-analyze verify /etc/systemd/system/eml-sensor.service
 2sudo systemctl daemon-reload
 3sudo systemctl enable --now eml-sensor.service
 4systemctl status eml-sensor.service
 5journalctl -u eml-sensor.service -f
 6systemctl show eml-sensor.service -p ActiveState -p SubState -p NRestarts

A maintainable application separates six concerns:

Configuration should be checked before the application begins useful work. Reject unknown units, impossible ranges, malformed addresses, missing files, and mutually inconsistent options. A default is safe only if its consequences are understood. Credentials should not appear in command-line arguments because other users or diagnostic tools may expose them.

Logs should communicate events and context rather than produce noise. Include timestamp, severity, component, event, and relevant identifiers. Avoid secrets and unnecessary personal data. Rate-limit repeated errors. Logging every high-rate sensor sample into the system journal may exhaust storage; measurement data and operational logs have different purposes.

Correct time supports certificate validation, event ordering, scheduled work, distributed logs, and incident reconstruction. Linux distinguishes system time from a hardware real-time clock, and wall-clock time from monotonic time. Wall time can jump when corrected; monotonic time is suitable for durations and timeouts.

At boot, a board without a reliable hardware clock may temporarily believe an old date. Applications should represent whether timestamps are synchronized and avoid silently treating uncertain time as valid. Inspect with:

1timedatectl status
2date --iso-8601=seconds
3systemctl status systemd-timesyncd
4journalctl -u systemd-timesyncd --since today

Use a systemd timer or a deliberate scheduler for periodic administrative work. A continuously running acquisition service should usually control its own sampling schedule using monotonic timing rather than repeatedly starting from cron. Prevent overlapping jobs when execution may exceed the interval.

Manual copying is acceptable for a first experiment but becomes unreliable when several devices or students are involved. A deployment procedure should identify the intended target, verify prerequisites, install a versioned release, validate configuration, restart safely, perform a health check, and preserve a rollback path.

A practical release layout is:

1/opt/eml/releases/2026-09-06-a1b2c3d/
2/opt/eml/releases/2026-09-13-d4e5f6a/
3/opt/eml/current -> /opt/eml/releases/2026-09-13-d4e5f6a/

Replacing the active symbolic link with a prepared link using a same-file-system rename makes the release switch atomic. A remove-then-create sequence leaves a gap and is not an atomic switch. Rollback must preserve a compatible previous executable, configuration, data schema, permissions, and service definition. If an update irreversibly migrates stored data, replacing the executable alone is not a rollback.

A process may exist while producing no useful result. Liveness asks whether the component is making progress and should remain running. Readiness asks whether it can currently accept useful work. A service can be live but not ready, for example while waiting for a sensor or reconnecting to a broker.

Health evidence can include:

• timestamp and sequence number of the last valid measurement;
• ability to open and communicate with the required device;
• backlog depth and age of the oldest unsent record;
• available disk space and writable data partition;
• successful completion of an application self-test; and
• externally observed output, not only an internal “healthy” flag.

Restart policy must match failure semantics. Restarting after an unexpected software exit is reasonable. Rapidly restarting with invalid configuration can create a loop that fills logs and hides the root cause. Apply backoff and start limits, expose the persistent failure, and define when human intervention is required.

A hardware watchdog can reset a completely unresponsive machine, but a reset is not proof of recovery. The system must return to a safe state, preserve diagnostic evidence, and avoid repeatedly actuating hardware during a reboot loop.

These questions assess Module 2 and prepare students for embedded Linux, IoT, robotics, DevOps, and systems-engineering interviews. Strong answers state assumptions, compare alternatives, and describe evidence. Attempt the questions before reading the answer key.

Questions

Answers and discussion