New approaches for embedded development

The idea for this posting started when I read New approaches to dominate in embedded development article. Then I found some ther related articles and here is the result: long article.

Embedded devices, or embedded systems, are specialized computer systems that constitute components of larger electromechanical systems with which they interface. The advent of low-cost wireless connectivity is altering many things in embedded development: With a connection to the Internet, an embedded device can gain access to essentially unlimited processing power and memory in cloud service – and at the same time you need to worry about communication issues like breaks connections, latency and security issues.

Those issues are espcecially in the center of the development of popular Internet of Things device and adding connectivity to existing embedded systems. All this means that the whole nature of the embedded development effort is going to change. A new generation of programmers are already making more and more embedded systems. Rather than living and breathing C/C++, the new generation prefers more high-level, abstract languages (like Java, Python, JavaScript etc.). Instead of trying to craft each design to optimize for cost, code size, and performance, the new generation wants to create application code that is separate from an underlying platform that handles all the routine details. Memory is cheap, so code size is only a minor issue in many applications.

Historically, a typical embedded system has been designed as a control-dominated system using only a state-oriented model, such as FSMs. However, the trend in embedded systems design in recent years has been towards highly distributed architectures with support for concurrency, data and control flow, and scalable distributed computations. For example computer networks, modern industrial control systems, electronics in modern car,Internet of Things system fall to this category. This implies that a different approach is necessary.

Companies are also marketing to embedded developers in new ways. Ultra-low cost development boards to woo makers, hobbyists, students, and entrepreneurs on a shoestring budget to a processor architecture for prototyping and experimentation have already become common.If you look under the hood of any connected embedded consumer or mobile device, in addition to the OS you will find a variety of middleware applications. As hardware becomes powerful and cheap enough that the inefficiencies of platform-based products become moot. Leaders with Embedded systems development lifecycle management solutions speak out on new approaches available today in developing advanced products and systems.

Traditional approaches

C/C++

Tradionally embedded developers have been living and breathing C/C++. For a variety of reasons, the vast majority of embedded toolchains are designed to support C as the primary language. If you want to write embedded software for more than just a few hobbyist platforms, your going to need to learn C. Very many embedded systems operating systems, including Linux Kernel, are written using C language. C can be translated very easily and literally to assembly, which allows programmers to do low level things without the restrictions of assembly. When you need to optimize for cost, code size, and performance the typical choice of language is C. Still C is today used for maximum efficiency instead of C++.

C++ is very much alike C, with more features, and lots of good stuff, while not having many drawbacks, except fror it complexity. The had been for years suspicion C++ is somehow unsuitable for use in small embedded systems. At some time many 8- and 16-bit processors were lacking a C++ compiler, that may be a concern, but there are now 32-bit microcontrollers available for under a dollar supported by mature C++ compilers.Today C++ is used a lot more in embedded systems. There are many factors that may contribute to this, including more powerful processors, more challenging applications, and more familiarity with object-oriented languages.

And if you use suitable C++ subset for coding, you can make applications that work even on quite tiny processors, let the Arduino system be an example of that: You’re writing in C/C++, using a library of functions with a fairly consistent API. There is no “Arduino language” and your “.ino” files are three lines away from being standard C++.

Today C++ has not displaced C. Both of the languages are widely used, sometimes even within one system – for example in embedded Linux system that runs C++ application. When you write a C or C++ programs for modern Embedded Linux you typically use GCC compiler toolchain to do compilation and make file to manage compilation process.

Most organization put considerable focus on software quality, but software security is different. When the security is very much talked about topic todays embedded systems, the security of the programs written using C/C++ becomes sometimes a debated subject. Embedded development presents the challenge of coding in a language that’s inherently insecure; and quality assurance does little to ensure security. The truth is that majority of today’s Internet connected systems have their networking fuctionality written using C even of the actual application layer is written using some other methods.

Java

Java is a general-purpose computer programming language that is concurrent, class-based and object-oriented.The language derives much of its syntax from C and C++, but it has fewer low-level facilities than either of them. Java is intended to let application developers “write once, run anywhere” (WORA), meaning that compiled Java code can run on all platforms that support Java without the need for recompilation.Java applications are typically compiled to bytecode that can run on any Java virtual machine (JVM) regardless of computer architecture. Java is one of the most popular programming languages in use, particularly for client-server web applications. In addition to those it is widely used in mobile phones (Java apps in feature phones,) and some embedded applications. Some common examples include SIM cards, VOIP phones, Blu-ray Disc players, televisions, utility meters, healthcare gateways, industrial controls, and countless other devices.

Some experts point out that Java is still a viable option for IoT programming. Think of the industrial Internet as the merger of embedded software development and the enterprise. In that area, Java has a number of key advantages: first is skills – there are lots of Java developers out there, and that is an important factor when selecting technology. Second is maturity and stability – when you have devices which are going to be remotely managed and provisioned for a decade, Java’s stability and care about backwards compatibility become very important. Third is the scale of the Java ecosystem – thousands of companies already base their business on Java, ranging from Gemalto using JavaCard on their SIM cards to the largest of the enterprise software vendors.

Although in the past some differences existed between embedded Java and traditional PC based Java solutions, the only difference now is that embedded Java code in these embedded systems is mainly contained in constrained memory, such as flash memory. A complete convergence has taken place since 2010, and now Java software components running on large systems can run directly with no recompilation at all on design-to-cost mass-production devices (consumers, industrial, white goods, healthcare, metering, smart markets in general,…) Java for embedded devices (Java Embedded) is generally integrated by the device manufacturers. It is NOT available for download or installation by consumers. Originally Java was tightly controlled by Sun (now Oracle), but in 2007 Sun relicensed most of its Java technologies under the GNU General Public License. Others have also developed alternative implementations of these Sun technologies, such as the GNU Compiler for Java (bytecode compiler), GNU Classpath (standard libraries), and IcedTea-Web (browser plugin for applets).

My feelings with Java is that if your embedded systems platform supports Java and you know hot to code Java, then it could be a good tool. If your platform does not have ready Java support, adding it could be quite a bit of work.

 

Increasing trends

Databases

Embedded databases are coming more and more to the embedded devices. If you look under the hood of any connected embedded consumer or mobile device, in addition to the OS you will find a variety of middleware applications. One of the most important and most ubiquitous of these is the embedded database. An embedded database system is a database management system (DBMS) which is tightly integrated with an application software that requires access to stored data, such that the database system is “hidden” from the application’s end-user and requires little or no ongoing maintenance.

There are many possible databases. First choice is what kind of database you need. The main choices are SQL databases and simpler key-storage databases (also called NoSQL).

SQLite is the Database chosen by virtually all mobile operating systems. For example Android and iOS ship with SQLite. It is also built into for example Firefox web browser. It is also often used with PHP. So SQLite is probably a pretty safe bet if you need relational database for an embedded system that needs to support SQL commands and does not need to store huge amounts of data (no need to modify database with millions of lines of data).

If you do not need relational database and you need very high performance, you need probably to look somewhere else.Berkeley DB (BDB) is a software library intended to provide a high-performance embedded database for key/value data. Berkeley DB is written in Cwith API bindings for many languages. BDB stores arbitrary key/data pairs as byte arrays. There also many other key/value database systems.

RTA (Run Time Access) gives easy runtime access to your program’s internal structures, arrays, and linked-lists as tables in a database. When using RTA, your UI programs think they are talking to a PostgreSQL database (PostgreSQL bindings for C and PHP work, as does command line tool psql), but instead of normal database file you are actually accessing internals of your software.

Software quality

Building quality into embedded software doesn’t happen by accident. Quality must be built-in from the beginning. Software startup checklist gives quality a head start article is a checklist for embedded software developers to make sure they kick-off their embedded software implementation phase the right way, with quality in mind

Safety

Traditional methods for achieving safety properties mostly originate from hardware-dominated systems. Nowdays more and more functionality is built using software – including safety critical functions. Software-intensive embedded systems require new approaches for safety. Embedded Software Can Kill But Are We Designing Safely?

IEC, FDA, FAA, NHTSA, SAE, IEEE, MISRA, and other professional agencies and societies work to create safety standards for engineering design. But are we following them? A survey of embedded design practices leads to some disturbing inferences about safety.Barr Group’s recent annual Embedded Systems Safety & Security Survey indicate that we all need to be concerned: Only 67 percent are designing to relevant safety standards, while 22 percent stated that they are not—and 11 percent did not even know if they were designing to a standard or not.

If you were the user of a safety-critical embedded device and learned that the designers had not followed best practices and safety standards in the design of the device, how worried would you be? I know I would be anxious, and quite frankly. This is quite disturbing.

Security

The advent of low-cost wireless connectivity is altering many things in embedded development – it has added to your list of worries need to worry about communication issues like breaks connections, latency and security issues. Understanding security is one thing; applying that understanding in a complete and consistent fashion to meet security goals is quite another. Embedded development presents the challenge of coding in a language that’s inherently insecure; and quality assurance does little to ensure security.

Developing Secure Embedded Software white paper  explains why some commonly used approaches to security typically fail:

MISCONCEPTION 1: SECURITY BY OBSCURITY IS A VALID STRATEGY
MISCONCEPTION 2: SECURITY FEATURES EQUAL SECURE SOFTWARE
MISCONCEPTION 3: RELIABILITY AND SAFETY EQUAL SECURITY
MISCONCEPTION 4: DEFENSIVE PROGRAMMING GUARANTEES SECURITY

Many organizations are only now becoming aware of the need to incorporate security into their software development lifecycle.

Some techniques for building security to embedded systems:

Use secure communications protocols and use VPN to secure communications
The use of Public Key Infrastructure (PKI) for boot-time and code authentication
Establishing a “chain of trust”
Process separation to partition critical code and memory spaces
Leveraging safety-certified code
Hardware enforced system partitioning with a trusted execution environment
Plan the system so that it can be easily and safely upgraded when needed

Flood of new languages

Rather than living and breathing C/C++, the new generation prefers more high-level, abstract languages (like Java, Python, JavaScript etc.). So there is a huge push to use interpreted and scripting also in embedded systems. Increased hardware performance on embedded devices combined with embedded Linux has made the use of many scripting languages good tools for implementing different parts of embedded applications (for example web user interface). Nowadays it is common to find embedded hardware devices, based on Raspberry Pi for instance, that are accessible via a network, run Linux and come with Apache and PHP installed on the device.  There are also many other relevant languages

One workable solution, especially for embedded Linux systems is that part of the activities organized by totetuettu is a C program instead of scripting languages ​​(Scripting). This enables editing operation simply script files by editing without the need to turn the whole system software again.  Scripting languages ​​are also tools that can be implemented, for example, a Web user interface more easily than with C / C ++ language. An empirical study found scripting languages (such as Python) more productive than conventional languages (such as C and Java) for a programming problem involving string manipulation and search in a dictionary.

Scripting languages ​​have been around for a couple of decades Linux and Unix server world standard tools. the proliferation of embedded Linux and resources to merge systems (memory, processor power) growth has made them a very viable tool for many embedded systems – for example, industrial systems, telecommunications equipment, IoT gateway, etc . Some of the command language is suitable for up well even in quite small embedded environments.
I have used with embedded systems successfully mm. Bash, AWK, PHP, Python and Lua scripting languages. It works really well and is really easy to make custom code quickly .It doesn’t require a complicated IDE; all you really need is a terminal – but if you want there are many IDEs that can be used.
High-level, dynamically typed languages, such as Python, Ruby and JavaScript. They’re easy—and even fun—to use. They lend themselves to code that easily can be reused and maintained.

There are some thing that needs to be considered when using scripting languages. Sometimes lack of static checking vs a regular compiler can cause problems to be thrown at run-time. But it is better off practicing “strong testing” than relying on strong typing. Other ownsides of these languages is that they tend to execute more slowly than static languages like C/C++, but for very many aplications they are more than adequate. Once you know your way around dynamic languages, as well the frameworks built in them, you get a sense of what runs quickly and what doesn’t.

Bash and other shell scipting

Shell commands are the native language of any Linux system. With the thousands of commands available for the command line user, how can you remember them all? The answer is, you don’t. The real power of the computer is its ability to do the work for you – the power of the shell script is the way to easily to automate things by writing scripts. Shell scripts are collections of Linux command line commands that are stored in a file. The shell can read this file and act on the commands as if they were typed at the keyboard.In addition to that shell also provides a variety of useful programming features that you are familar on other programming langauge (if, for, regex, etc..). Your scripts can be truly powerful. Creating a script extremely straight forward: It can be created by opening a separate editor such or you can do it through a terminal editor such as VI (or preferably some else more user friendly terminal editor). Many things on modern Linux systems rely on using scripts (for example starting and stopping different Linux services at right way).

One of the most useful tools when developing from within a Linux environment is the use of shell scripting. Scripting can help aid in setting up environment variables, performing repetitive and complex tasks and ensuring that errors are kept to a minimum. Since scripts are ran from within the terminal, any command or function that can be performed manually from a terminal can also be automated!

The most common type of shell script is a bash script. Bash is a commonly used scripting language for shell scripts. In BASH scripts (shell scripts written in BASH) users can use more than just BASH to write the script. There are commands that allow users to embed other scripting languages into a BASH script.

There are also other shells. For example many small embedded systems use BusyBox. BusyBox providesis software that provides several stripped-down Unix tools in a single executable file (more than 300 common command). It runs in a variety of POSIX environments such as Linux, Android and FreeeBSD. BusyBox become the de facto standard core user space toolset for embedded Linux devices and Linux distribution installers.

Shell scripting is a very powerful tool that I used a lot in Linux systems, both embedded systems and servers.

Lua

Lua is a lightweight  cross-platform multi-paradigm programming language designed primarily for embedded systems and clients. Lua was originally designed in 1993 as a language for extending software applications to meet the increasing demand for customization at the time. It provided the basic facilities of most procedural programming languages. Lua is intended to be embedded into other applications, and provides a C API for this purpose.

Lua has found many uses in many fields. For example in video game development, Lua is widely used as a scripting language by game programmers. Wireshark network packet analyzer allows protocol dissectors and post-dissector taps to be written in Lua – this is a good way to analyze your custom protocols.

There are also many embedded applications. LuCI, the default web interface for OpenWrt, is written primarily in Lua. NodeMCU is an open source hardware platform, which can run Lua directly on the ESP8266 Wi-Fi SoC. I have tested NodeMcu and found it very nice system.

PHP

PHP is a server-side HTML embedded scripting language. It provides web developers with a full suite of tools for building dynamic websites but can also be used as a general-purpose programming language. Nowadays it is common to find embedded hardware devices, based on Raspberry Pi for instance, that are accessible via a network, run Linux and come with Apache and PHP installed on the device. So on such enviroment is a good idea to take advantage of those built-in features for the applications they are good – for building web user interface. PHP is often embedded into HTML code, or it can be used in combination with various web template systems, web content management system and web frameworks. PHP code is usually processed by a PHP interpreter implemented as a module in the web server or as a Common Gateway Interface (CGI) executable.

Python

Python is a widely used high-level, general-purpose, interpreted, dynamic programming language. Its design philosophy emphasizes code readability. Python interpreters are available for installation on many operating systems, allowing Python code execution on a wide variety of systems. Many operating systems include Python as a standard component; the language ships for example with most Linux distributions.

Python is a multi-paradigm programming language: object-oriented programming and structured programming are fully supported, and there are a number of language features which support functional programming and aspect-oriented programming,  Many other paradigms are supported using extensions, including design by contract and logic programming.

Python is a remarkably powerful dynamic programming language that is used in a wide variety of application domains. Since 2003, Python has consistently ranked in the top ten most popular programming languages as measured by the TIOBE Programming Community Index. Large organizations that make use of Python include Google, Yahoo!, CERN, NASA. Python is used successfully in thousands of real world business applications around globally, including many large and mission-critical systems such as YouTube.com and Google.com.

Python was designed to be highly extensible. Libraries like NumPy, SciPy and Matplotlib allow the effective use of Python in scientific computing. Python is intended to be a highly readable language. Python can also be embedded in existing applications and hasbeen successfully embedded in a number of software products as a scripting language. Python can serve as a scripting language for web applications, e.g., via mod_wsgi for the Apache web server.

Python can be used in embedded, small or minimal hardware devices. Some modern embedded devices have enough memory and a fast enough CPU to run a typical Linux-based environment, for example, and running CPython on such devices is mostly a matter of compilation (or cross-compilation) and tuning. Various efforts have been made to make CPython more usable for embedded applications.

For more limited embedded devices, a re-engineered or adapted version of CPython, might be appropriateExamples of such implementations include the following: PyMite, Tiny Python, Viper. Sometimes the embedded environment is just too restrictive to support a Python virtual machine. In such cases, various Python tools can be employed for prototyping, with the eventual application or system code being generated and deployed on the device. Also MicroPython and tinypy have been ported Python to various small microcontrollers and architectures. Real world applications include Telit GSM/GPRS modules that allow writing the controlling application directly in a high-level open-sourced language: Python.

Python on embedded platforms? It is quick to develop apps, quick to debug – really easy to make custom code quickly. Sometimes lack of static checking vs a regular compiler can cause problems to be thrown at run-time. To avoid those try to have 100% test coverage. pychecker is a very useful too also which will catch quite a lot of common errors. The only downsides for embedded work is that sometimes python can be slow and sometimes it uses a lot of memory (relatively speaking). An empirical study found scripting languages (such as Python) more productive than conventional languages (such as C and Java) for a programming problem involving string manipulation and search in a dictionary. Memory consumption was often “better than Java and not much worse than C or C++”.

JavaScript and node.js

JavaScript is a very popular high-level language. Love it or hate it, JavaScript is a popular programming language for many, mainly because it’s so incredibly easy to learn. JavaScript’s reputation for providing users with beautiful, interactive websites isn’t where its usefulness ends. Nowadays, it’s also used to create mobile applications, cross-platform desktop software, and thanks to Node.js, it’s even capable of creating and running servers and databases!  There is huge community of developers. JavaScript is a high-level language.

Its event-driven architecture fits perfectly with how the world operates – we live in an event-driven world. This event-driven modality is also efficient when it comes to sensors.

Regardless of the obvious benefits, there is still, understandably, some debate as to whether JavaScript is really up to the task to replace traditional C/C++ software in Internet connected embedded systems.

It doesn’t require a complicated IDE; all you really need is a terminal.

JavaScript is a high-level language. While this usually means that it’s more human-readable and therefore more user-friendly, the downside is that this can also make it somewhat slower. Being slower definitely means that it may not be suitable for situations where timing and speed are critical.

JavaScript is already in embedded boards. You can run JavaScipt on Raspberry Pi and BeagleBone. There are also severa other popular JavaScript-enabled development boards to help get you started: The Espruino is a small microcontroller that runs JavaScript. The Tessel 2 is a development board that comes with integrated wi-fi, an ethernet port, two USB ports, and companion source library downloadable via the Node Package Manager. The Kinoma Create, dubbed the “JavaScript powered Internet of Things construction kit.”The best part is that, depending on the needs of your device, you can even compile your JavaScript code into C!

JavaScript for embedded systems is still in its infancy, but we suspect that some major advancements are on the horizon.We for example see a surprising amount of projects using Node.js.Node.js is an open-source, cross-platform runtime environment for developing server-side Web applications. Node.js has an event-driven architecture capable of asynchronous I/O that allows highly scalable servers without using threading, by using a simplified model of event-driven programming that uses callbacks to signal the completion of a task. The runtime environment interprets JavaScript using Google‘s V8 JavaScript engine.Node.js allows the creation of Web servers and networking tools using JavaScript and a collection of “modules” that handle various core functionality. Node.js’ package ecosystem, npm, is the largest ecosystem of open source libraries in the world. Modern desktop IDEs provide editing and debugging features specifically for Node.js applications

JXcore is a fork of Node.js targeting mobile devices and IoTs. JXcore is a framework for developing applications for mobile and embedded devices using JavaScript and leveraging the Node ecosystem (110,000 modules and counting)!

Why is it worth exploring node.js development in an embedded environment? JavaScript is a widely known language that was designed to deal with user interaction in a browser.The reasons to use Node.js for hardware are simple: it’s standardized, event driven, and has very high productivity: it’s dynamically typed, which makes it faster to write — perfectly suited for getting a hardware prototype out the door. For building a complete end-to-end IoT system, JavaScript is very portable programming system. Typically an IoT projects require “things” to communicate with other “things” or applications. The huge number of modules available in Node.js makes it easier to generate interfaces – For example, the HTTP module allows you to create easily an HTTP server that can easily map the GET method specific URLs to your software function calls. If your embedded platform has ready made Node.js support available, you should definately consider using it.

Future trends

According to New approaches to dominate in embedded development article there will be several camps of embedded development in the future:

One camp will be the traditional embedded developer, working as always to craft designs for specific applications that require the fine tuning. These are most likely to be high-performance, low-volume systems or else fixed-function, high-volume systems where cost is everything.

Another camp might be the embedded developer who is creating a platform on which other developers will build applications. These platforms might be general-purpose designs like the Arduino, or specialty designs such as a virtual PLC system.

This third camp is likely to become huge: Traditional embedded development cannot produce new designs in the quantities and at the rate needed to deliver the 50 billion IoT devices predicted by 2020.

Transition will take time. The enviroment is different than computer and mobile world. There are too many application areas with too widely varying requirements for a one-size-fits-all platform to arise.

But the shift will happen as hardware becomes powerful and cheap enough that the inefficiencies of platform-based products become moot.

 

Sources

Most important information sources:

New approaches to dominate in embedded development

A New Approach for Distributed Computing in Embedded Systems

New Approaches to Systems Engineering and Embedded Software Development

Lua (programming language)

Embracing Java for the Internet of Things

Node.js

Wikipedia Node.js

Writing Shell Scripts

Embedded Linux – Shell Scripting 101

Embedded Linux – Shell Scripting 102

Embedding Other Languages in BASH Scripts

PHP Integration with Embedded Hardware Device Sensors – PHP Classes blog

PHP

Python (programming language)

JavaScript: The Perfect Language for the Internet of Things (IoT)

Node.js for Embedded Systems

Embedded Python

MicroPython – Embedded Pytho

Anyone using Python for embedded projects?

Telit Programming Python

JavaScript: The Perfect Language for the Internet of Things (IoT)

MICROCONTROLLERS AND NODE.JS, NATURALLY

Node.js for Embedded Systems

Why node.js?

Node.JS Appliances on Embedded Linux Devices

The smartest way to program smart things: Node.js

Embedded Software Can Kill But Are We Designing Safely?

DEVELOPING SECURE EMBEDDED SOFTWARE

 

 

 

1,686 Comments

  1. Tomi Engdahl says:

    The Whole Thing In Python
    https://hackaday.com/2023/01/02/the-whole-thing-in-python/

    [hsgw] built a macropad in Python, and that’s not a strange language to choose to program the firmware in these days. But that’s just the tip of the iceberg. The whole process — from schematic capture, through routing and generating the PCB, and even extending to making the case — was done programmatically, in Python.

    Keyboard as a Python Code
    A keyboard designed by python only.
    https://hackaday.io/project/188907-keyboard-as-a-python-code

    Reply
  2. Tomi Engdahl says:

    The GitHub Silverware Drawer Dilemma, Or: Finding Active Repository Forks
    https://hackaday.com/2023/01/08/the-github-silverware-drawer-dilemma-or-finding-active-repository-forks/

    An fortunate reality of GitHub and similar sites is that projects that are abandoned by the maintainer are often continued by someone else who forked the project. Unfortunately, the ease of forking also means that GitHub projects tend to have a lot of forks, with the popular projects having hundreds of them. Since GitHub has elected to not provide a way to filter or sort these forks, finding the most active fork can be rather harrowing.

    In addition, a popular project’s dead repository tends to score higher in search results than replacement forks.

    Meanwhile, the Active Forks project by [Samar Dhwoj Acharya] provides a sortable list of project forks when provided with a GitHub repository name. This helps enormously when trying to find the freshest forks in a whole list. This is similar to the Useful Forks project that provides a web-based interface in addition to a Chrome extension.

    https://github.com/techgaun/active-forks

    Reply
  3. Tomi Engdahl says:

    https://m.facebook.com/story.php?story_fbid=pfbid0354D9LEyX7wmgfNY4frEEjq1xWHZKUeLe8SmprypPcp4ZbSbnBfNpVhc3VGLnCPTMl&id=164806146894658

    As a platform to serve a community of developers, we value the documentation as much as we do for the product. Our AE team has created 100+ wikis corresponding with our new released products in 2022. Here are the top 10 most read wikis in 2022, which include topics of LVGL UI, Bluetooth for Nordic Semiconductor’s nRF52 Boards, tinyML, mmWave Sensor, Motion recognition, object detection with Ultralytics YOLOv5 and Roboflow.

    Get the full list here: https://bit.ly/top10wiki

    #machinelearning #YOLOv5 #tinyML #LVGL #wiki

    Reply
  4. Tomi Engdahl says:

    How to Perform Firmware Updates Without Halting Firmware Execution
    https://www.digikey.com/en/articles/how-to-perform-firmware-updates-without-halting-firmware-execution?dclid=CPyIt-CczvwCFRuT_QcdEhsDBg

    nternet of Things (IoT) sensor-based applications are expanding, and so too is the size and complexity of the microcontroller firmware in the IoT endpoint. This firmware must become more efficient to speed execution, which is one reason flash firmware updates in the field are a necessity. However, securely updating firmware in the field usually requires halting execution of firmware while the update is in progress. Depending on the architecture, the size of the update, and network speed, this can be accomplished in as quickly as a minute or as long as an hour. For critical applications this delay can be unacceptable.

    This article explains the considerations for updating interrupt-driven firmware in the field and the need to keep executing application firmware while the update is in process. It then introduces the PIC32MZ2048EFH144T-I/PH microcontroller from Microchip Technology and shows how it can be used to execute firmware while simultaneously receiving updated firmware over a network.

    Reply
  5. Tomi Engdahl says:

    Microcontrollers are limited to certain tasks and typically lack the ability for multithreading, which would allow for more than one user or task at a time without requiring multiple copies of a program or computer. Generally, microcontrollers cannot break larger or more complex applications into multiple threads.
    In this tutorial, Engineers Garage will show you how to schedule embedded tasks in Arduino using FreeRTOS: https://fal.cn/3vdfK

    Reply
  6. Tomi Engdahl says:

    Cloudification Demands New Approach to Timing Design
    December 23, 2022 Deepak Tripathi and Gary Giust
    https://www.eetimes.eu/cloudification-demands-new-approach-to-timing-design/?utm_source=newsletter&utm_campaign=link&utm_medium=EETimesEuropeWeekly-20230119

    The cloudification trend is putting unprecedented pressure on the crucial timing systems that regulate the high-frequency operation of devices such as servers, network switches, and routers. How should computing- and network-equipment manufacturers respond?

    Reply
  7. Tomi Engdahl says:

    QT tutki: Sulautettujen kehittäjät hakevat uutta tehoa
    https://www.uusiteknologia.fi/2023/01/31/qt-tutki-sulautettujen-kehittajat-hakevat-uutta-tehoa/

    Sijainti
    Etusivu > Artikkelit/raportit > QT tutki: Sulautettujen kehittäjät hakevat uutta tehoa
    QT tutki: Sulautettujen kehittäjät hakevat uutta tehoa

    Artikkelit/raportit

    - 31.1.2023

    Suomalaisen kehitystyökaluja kehittävän Qt Groupin teettämän selvityksen mukaan yli kolmannes IoT-laitteiden valmistajista kertoo tuotteiden markkinoille tulon viivästyneen ja kallistuneen. Apua haetaan nyt yrityksen mukaan monialustaisista ohjelmistotyökaluista ja uudesta osaamisesta.

    Brittiläisen markkinatutkimusyhtiö Censuswiden suomalaiselle Qt Groupille tuottamaan kyselytutkimukseen vastasi 250 sulautettujen laitteiden valmistajaa Yhdysvalloista, Isosta-Britanniasta, Ranskasta ja Saksasta.

    Tutkimuksen mukaan makrotalouden haasteet aiheuttavat viivästyksiä ja hintojen nousua IoT-laitteisiin. Suuri osa vastanneista (38 %) oli kärsinyt viivästyksistä tuotteiden tuomisessa markkinoille, kun taas lähes puolet (48 %) kertoi joutuneensa nostamaan tuotteidensa hintoja.

    ”Viime vuosien epävarma globaali taloustilanne on aiheuttanut merkittäviä häiriöitä useilla toimialoilla, joten ymmärrettävästi yritykset etsivät nyt keinoja tuottavuutensa parantamiseen”, sanoo ohjelmistotyökaluja tuottavan Qt Groupin Senior Vice President Marko Kaasila.

    Reply
  8. Tomi Engdahl says:

    Uusi alusta voisi olla ”IoT-laitteiden Android”
    https://etn.fi/index.php/13-news/14538-uusi-alusta-voisi-olla-iot-laitteiden-android

    Veteraani Ian Drewn perustama Foundries.IO yrittää tehdä vallankumousta sulautetussa kehityksessä. Tuote on universaali Linux-alusta, joka toimii millä tahansa piirisarjalla, missä tahansa pilvessä ja jossa tietoturvallinen koodin päivitys ja ylläpito olisi itsestään selvää.

    Alusta on nimeltään FoundriesFactory. Nimi ei ole kovin tarttuva, mutta ehkä sen pitäisi olla, sillä alusta pyrkii olemaan eräänlainen IoT- ja edge-laitteiden Windows tai Android. Universaali alusta. – Ehkä vertailu IoT-laitteiden ”RedHatiksi” on osuvampi, koska FoundriesFactory ei ole vain käyttöjärjestelmä, Drew tarkentaa. Nimityksen IoT-laitteiden Androidksi hän kyllä ymmärtää ja hyväksyy.

    Sanalla sanoen sulautettu markkina on aivan liian fragmentoitunut. – PC-koneissa on ehkä kymmenen eri suunnittelua, palvelimissa kaksi ja älypuhelimissa kymmeniä, mutta sulautetuissa satatuhatta eri suunnittelua. Jokainen tekee oman käyttöjärjestelmänsä, mikä on älytöntä.

    Jos olisi yksi universaali alusta, joka toimisi jokaisella raudalla. Ja lisäksi vielä kehitys onnistuisi pilvessä. Kehityksen mallia Drew kutsuu tutulla nimellä devops. Moni sulautettu kehittäjä pelkää nimeä tosissaan.

    - Kun kerron yrityksessä, että IoT-ohjelmistonkehityksen voi tehdä 10 kertaa nykyistä tehokkaammin ja paremmin, ja vielä alusta asti tietoturvallisesti, tottakai se herättää ajatuksia, Drew myöntää.

    Mutta miksi näin ei ole tehty? – Koska se on pahuksen vaikeaa, Drew sanoo.

    Foundries.IO valitsi alusta asti OTA-päivitystensä mekanismiksi TUF-kehyksen. (The Update Framework). Se on alusta asti rakennettu kestämään jopa valtiollisten toimijoiden hyökkäykset. TUF estää koodin väärennöksen, vaikka salausavaimiin olisi päästy käsiksi.

    Reply
  9. Tomi Engdahl says:

    Find SWD Points Quickly, No Extra Hardware Needed
    https://hackaday.com/2023/01/31/find-swd-points-quickly-no-extra-hardware-needed/

    Say you’re tinkering with a smart device powered by a CPU that uses Serial Wire Debug (SWD), but doesn’t mark the testpoints. Finding SWD on a board — how hard could it be? With [Aaron Christophel]’s method, you can find the SWD interface on a PCB within a few minutes’ time. All you need is two needles, a known-to-be-ground connection, an SWD dongle of some kind, and a computer with an audio output. What’s best — you could easily transfer the gist of this method to other programming interface types!

    The idea is simple: you wire the ground up, connect the needles to SWDIO and SWCLK, launch [Aaron]’s Python script, then start poking around all the unnamed test points. The script runs JLink software to probe for SWD devices attached to the probes — if an SWD interface isn’t found, it beeps idly, but as soon as the device is detected, your computer will start beeping at you in a lively manner.

    Quick tip finding an unknown Debug interface pinout (SWD) shown on the MiBand with
    https://www.youtube.com/watch?v=r6hGHZEtEE0

    Reply
  10. Tomi Engdahl says:

    PlatformIO: All you need to know in 10 Minutes!
    https://www.youtube.com/watch?v=PYSy_PLjytQ

    Tired of the Arduino IDE? Looking for quick and easy to use alternatives for your embedded coding? Interested in some nice code highlighting, autocomplete, Intellisense and fully customizable overlays? Code for right about any board: Arduino, ESP32, STM32 and more! Migrate to PlatformIO today and start coding your projects with more joy and less BS!

    Reply
  11. Tomi Engdahl says:

    Debugging an Arduino Sketch Running on an Uno
    https://www.youtube.com/watch?v=7Pm9_LtXdhM

    How to build a hardware debugger that allows you to debug Arduino programs running on an Arduino Uno or ATtiny in 4 easy steps.

    Table of Contents:
    00:26 – Motivation
    01:32 – GDB: The GNU Project Debugger
    01:44 – Embedded Debugging
    05:04 – Step 1: Install PlatformIO
    08:31 – Step 2: Install firmware for HW debugger
    11:08 – Step 3: Connect debuggger to target
    16:42 – Step 4: Start debugging

    A debugWIRE Hardware Debugger for Less Than €10
    https://hinterm-ziel.de/index.php/2022/01/13/a-debugwire-hardware-debugger-for-less-than-10-e/

    dw-probe: The Hardware for the Hardware Debugger
    https://hinterm-ziel.de/index.php/2022/01/04/dw-probe-the-hardware-for-the-hardware-debugger/

    Reply
  12. Tomi Engdahl says:

    Scratch Your Itch To Program A Microcontroller
    https://hackaday.com/2023/02/03/scratch-your-itch-to-program-a-microcontroller/

    One of the fun things about “old school” computers is that it was fairly easy to get kids into programming them. The old Basic interpreters were pretty forgiving, and you could do some clever things easily with very little theory or setup. These days, you are more likely to sneak kids into programming via Scratch — a system for setting up programs via blocks in a GUI. Again, you can get simple results simply. With Scratch or Basic, complex things have a way of turning out complex, but that’s to be expected. If you want to try a Scratch-inspired take on microcontroller programming, check out MicroBlocks. It will work with several common boards, including the micro:bit and the Raspberry Pi Pico. You can use it in a browser or download versions for Linux, Windows, Mac, or even Chromebooks.

    You can see a video below about the micro:bit version from a year ago. The tool is advancing, so you’ll find many new features compared to the video, but it will still give you an idea of what’s happening.

    https://microblocks.fun/

    Reply
  13. Tomi Engdahl says:

    Valvontakameran kuvaa valvoo yhä useammin tekoäly
    https://etn.fi/index.php/13-news/14554-valvontakameran-kuvaa-valvoo-yhae-useammin-tekoaely

    Tänä vuonna nähdään merkittävä kasvu AI-pohjaisen analytiikan käyttöönottoa kameroissa ja videonhallintajärjestelmissä. Kamerakuvaa on yksinkertaisesti liian paljon, jotta ihmiset voisivat valvoa niitä tehokkaasti. AI-pohjaisen analytiikan avulla turvallisuusosastot kykenevät tekemään enemmän vähemmillä resursseilla, arvioi turvallisuusalan anturiratkaisuja kehittävä i-PRO.

    Vuonna 2023 siirrytään videokuvan tallentamisesta tiedonkeruuseen. Haasteeksi muodostuu se, miten tehokkaasti organisaatiot kykenevät hyödyntämään näitä tietoja, ei pelkästään turvallisuutta silmälläpitäen, vaan myös eri osastojen välisissä toimissa tehokkuuden ja liikevaihdon lisäämiseksi.

    Prosessoinnista verkon reunalla tulee yleisempää, kun reunan laitteista itsestään tulee alati tehokkaampia. Uudet tavat valjastaa jaetun IoT:n tehoa konttiteknologioiden avulla jatkavat yleistymistään. Dockerin ja Kubernetesin kaltaiset konttiteknologiat sekä useiden laitteiden välillä jaettujen resurssien käsite tulee lisäämään prosessointinopeutta, parantamaan analytiikkalisäkkeiden integrointia ja nopeuttamaan laiteohjelmistojen päivityksiä ja uusien analytiikkaominaisuuksien käyttöönottoa.

    Reply
  14. Tomi Engdahl says:

    Suomalaisen BCB Medicalin ohjelmistolle tärkeä EU-sertifikaatti
    https://etn.fi/index.php/13-news/14553-suomalaisen-bcb-medicalin-ohjelmistolle-taerkeae-eu-sertifikaatti

    Turkulainen vuonna 2003 perustettu BCB Medical on yksi ensimmäisiä yrityksiä Suomessa, jonka ohjelmisto täyttää EU:n lääkinnällisten laitteiden vaatimukset. BCB Medicalille myönnetty MDR-sertifikaatti on arvokas tunnustus potilasturvallisuuden eteen tehdystä määrätietoisesta työstä.

    Pohjoismaiden johtava kliinisen datan analysointiyritys BCB Medical on saanut MDR-sertifikaatin BCB Disease Specific Register -ohjelmistolleen, jolla voidaan arvioida hoidon tehokkuutta ja laatua sekä parantaa hoitoketjuja. Sertifikaatti on Euroopan unionin lääkinnällisten laitteiden asetuksen eli MDR:n (Medical Devices Regulation) mukainen.

    Asetus tiukentaa EU:n vaatimuksia lääkinnällisille laitteille ja ohjelmistoille, yhtenäistää EU-tason sääntelyä ja parantaa potilasturvallisuutta. MDR-asetus on pakottava lainsäädäntö, joka koskee kaikkia lääkinnällisten laitteiden valmistajia.

    EU:n lääkinnällisten laitteiden MDR-asetus tuli voimaan 26.5.2021, ja siirtymäaikojen jälkeen myös aiemmin markkinoilla olevien lääkinnällisten laitteiden tulee täyttää MDR-sertifikaatin vaatimukset. BCB Medical on ensimmäisiä ohjelmistoyrityksiä Suomessa, joka lähti hakemaan uutta sertifikaattia.

    Reply
  15. Tomi Engdahl says:

    Using ChatGPT to Write Code for Arduino and ESP32
    https://dronebotworkshop.com/chatgpt/

    Today, we will use the amazing ChatGPT chatbot to write C++ and MicroPython code for both an Arduino Uno and an ESP32.

    Reply
  16. Tomi Engdahl says:

    THE CHEAPEST FLASH MICROCONTROLLER YOU CAN BUY IS ACTUALLY AN ARM CORTEX-M0+
    Puya’s 10-cent PY32 series is complicating the RISC-V narrative and has me doubting I’ll ever reach for an 8-bit part again.
    https://jaycarlson.net/2023/02/04/the-cheapest-flash-microcontroller-you-can-buy-is-actually-an-arm-cortex-m0/

    Browse the Microcontroller pages of LCSC and you’ll see tons of low-cost MCUs from companies like Padauk, Nyquest, Holychip, SimOne, and Fremont Micro Devices — with prices as low as 4 cents. The problem is these parts all use EPROM: Electronically Programmable Read Only Memory. The missing “E” means these aren’t electronically erasable — they’re OTP (One-Time Programmable) parts. Once you’ve burned your hex file, you’re locked in. OTP parts are widely used for simple devices in cost-sensitive applications, but they’re annoying to use in designs since to develop firmware, you need special emulator hardware (or a huge supply of ICs and a conveniently-located trash can).

    I was interested in finding the cheapest flash microcontroller LCSC sells, and it turns out it’s not a crusty old 8051 or a PIC16 clone — and it’s not the WCH CH32V003 that the Internet is freaking out about — it’s actually an Arm Cortex-M0+ made by Puya, it’s less than 10 cents in moderate quantities, and it’s awesome.

    PUYA PY32 SELECTION GUIDE
    Puya makes a range of ultra-low-cost Arm Cortex-M0+ parts in the PY32 series. There are three families in the series: the PY32F002A, the PY32F003, and the PY32F030. The families differ in memory, maximum speed, and peripheral inclusion, but they seem entirely progressive (e.g., firmware written for the PY32F002A should run on the PY32F003 and PY32F030). Regardless, all the parts are modern designs that use internal regulators and have an operating voltage range of 1.7 to 5.5V.

    Unlike many other low-cost MCUs that only come in one or two (usually low-density) package options, the PY32 has nine different choices, ranging from a tiny 2×3 mm DFN to a full-sized 32-pin LQFP — along with QFN, TSSOP, and drunk-soldering-friendly SOIC packages.

    I bought a ton of different PY32 parts from LCSC (just search for PY32). At the time of publication, prices have shot up quite a bit compared to when I first ordered these parts

    Reply
  17. Tomi Engdahl says:

    SCRATCH YOUR ITCH TO PROGRAM A MICROCONTROLLER
    https://hackaday.com/2023/02/03/scratch-your-itch-to-program-a-microcontroller/

    One of the fun things about “old school” computers is that it was fairly easy to get kids into programming them. The old Basic interpreters were pretty forgiving, and you could do some clever things easily with very little theory or setup. These days, you are more likely to sneak kids into programming via Scratch — a system for setting up programs via blocks in a GUI. Again, you can get simple results simply. With Scratch or Basic, complex things have a way of turning out complex, but that’s to be expected. If you want to try a Scratch-inspired take on microcontroller programming, check out MicroBlocks. It will work with several common boards, including the micro:bit and the Raspberry Pi Pico. You can use it in a browser or download versions for Linux, Windows, Mac, or even Chromebooks.

    Reply
  18. Tomi Engdahl says:

    Using ChatGPT to Write Code for Arduino and ESP32
    https://dronebotworkshop.com/chatgpt/

    Reply
  19. Tomi Engdahl says:

    How to Perform Firmware Updates Without Halting Firmware Execution
    https://www.digikey.com/en/articles/how-to-perform-firmware-updates-without-halting-firmware-execution?dclid=CLXE_-nmmf0CFfcIewodgpgNmQ

    Internet of Things (IoT) sensor-based applications are expanding, and so too is the size and complexity of the microcontroller firmware in the IoT endpoint. This firmware must become more efficient to speed execution, which is one reason flash firmware updates in the field are a necessity. However, securely updating firmware in the field usually requires halting execution of firmware while the update is in progress. Depending on the architecture, the size of the update, and network speed, this can be accomplished in as quickly as a minute or as long as an hour. For critical applications this delay can be unacceptable.

    This article explains the considerations for updating interrupt-driven firmware in the field and the need to keep executing application firmware while the update is in process. It then introduces the PIC32MZ2048EFH144T-I/PH microcontroller from Microchip Technology and shows how it can be used to execute firmware while simultaneously receiving updated firmware over a network.

    The importance of firmware updates

    Firmware is updated for four main reasons: to correct bugs in the code, to add or improve features, to make adjustments to system security, and to make firmware more efficient. Code efficiency is measured by the number of clock cycles it takes to perform a specific task or thread. The fewer clock cycles to perform a task, the more efficient the code, which speeds execution and usually (not always) reduces code size. This is especially true for IoT sensor-based endpoints as these applications are interrupt-driven and so must quickly switch context whenever a sensor or peripheral generates an interrupt.

    Two factors affecting the efficiency of interrupt-driven IoT applications are the efficiency of the architecture, and the efficiency of the code. While it is impractical to change the architecture of a microcontroller in the field, it is practical and normal to update the microcontroller firmware to improve efficiency.

    Sensor-based firmware is almost always interrupt-driven. Intelligent sensors connected to a microcontroller serial port can generate an interrupt to the microcontroller to halt normal execution so the firmware can vector to an interrupt service routine for that particular sensor. This is more efficient than sensors that need to be periodically polled to determine if the sensor reading has new data to transmit. The advantage of an interrupt-driven sensor strategy is that the microcontroller only spends clock cycles on reading the sensor when there is useful data to receive. Polling wastes clock cycles when firmware has to access the sensor to read data that is discarded because the sensor reading has not updated.

    With multiple sensors and tasks comes multiple subroutines and interrupt routines to manage them, increasing code size. Complex code requires some form of real-time operating system (RTOS) to manage all these separate tasks. The RTOS can be a simple firmware application written by the software engineer or an off-the-shelf product. The RTOS manages the different firmware tasks to make sure each individual task starts and finishes within the time necessary for the application to operate properly. If many tasks need to be managed by the RTOS, it is beneficial for the application for tasks to finish in as few clock cycles as possible. This prevents different tasks from delaying each other.

    Reply
  20. Tomi Engdahl says:

    Creating Bug-Free Software
    Tools like Rust and SPARK make creation of reliable software easier.
    https://www.electronicdesign.com/featured-media/video/webinar/webinar/21258224/creating-bugfree-software?utm_source=EG+ED+Connected+Solutions&utm_medium=email&utm_campaign=CPS230208022&o_eid=7211D2691390C9R&rdx.identpull=omeda|7211D2691390C9R&oly_enc_id=7211D2691390C9R

    Developing software is not an easy task especially keeping it bug-free for embedded applications like self-driving cars. There are many tools and techniques that can help make this happen from process oriented tools to programming languages like Rust and Ada/SPARK. We will be taking a look at some of these approaches including a discussion of NVIDIA’s security processor and software as well as the Muen separation kernel and Wookey Project.

    https://www.adacore.com/papers/nvidia-adoption-of-spark-new-era-in-security-critical-software-development
    https://muen.codelabs.ch/
    https://wookey-project.github.io/index.html

    Reply
  21. Tomi Engdahl says:

    Considering Semiconductor Implementation Aspects Early During Network-On-Chip Development
    facebook sharing buttontwitter sharing buttonlinkedin sharing buttonsharethis sharing button
    How implementation can fundamentally impact architecture.
    https://semiengineering.com/considering-semiconductor-implementation-aspects-early-during-network-on-chip-development/

    Reply
  22. Tomi Engdahl says:

    How to build better automotive applications with MIL testing
    https://www.edn.com/how-to-build-better-automotive-applications-with-mil-testing/

    It’s a busy day for the automotive team of a tier-1 company. Engineers are rushing to complete the modules assigned to them so that the unit testing can be performed. The engineers are worried about multiple testing iterations that would follow the module development. The deadline is fast approaching. It’s a moment of calm before the project manager goes into a tizzy and starts questioning the team about the delay. Oh, that’s a messy situation to be in.

    One good decision at the beginning of the project and all of this could have been avoided. If the automotive team had taken the model-based design approach, the situation would be much different.

    Reply
  23. Tomi Engdahl says:

    Puya PY32: The Cheapest Flash Microcontroller You Can Buy Is Actually An ARM Cortex-M0+
    https://hackaday.com/2023/02/16/puya-py32-the-cheapest-flash-microcontroller-you-can-buy-is-actually-an-arm-cortex-m0/

    There’s a bit of a contest going on when it comes to which is the cheapest microcontroller, yet most of the really cheap ones have one big trade-off in that they have one-time programmable (OTP) memory, generally requiring the use of an (expensive) device emulator during development. This raises the question of what the cheapest reprogrammable MCU is, which [Jay Carlson] postulates is found in the Puya PY32 ARM Cortex-M0+ based series.

    https://jaycarlson.net/2023/02/04/the-cheapest-flash-microcontroller-you-can-buy-is-actually-an-arm-cortex-m0/

    Reply
  24. Tomi Engdahl says:

    IoT-proton voi kehittää muutamassa minuutissa
    https://etn.fi/index.php/13-news/14657-iot-proton-voi-kehittaeae-muutamassa-minuutissa

    Renesasilla on saatu idea viedä IoT-laitteen kehitys pilveen ja erityisesti leikata aikaa ohjelmiston suunnittelusta. Tuloksena on Quick-Connect Studio -työkalu, jolla IoT-proton kehitys onnistuu parhaimmillaan muutamassa minuutissa.

    Tämän päivän kehityssykli on raskas projekti, Renesas muistuttaa. Insinöörit tutkivat ja määrittelevät projektin, keräävät laitetietoja ja vaatimuksia, asettelevat laitteistoja, kehittävät ohjelmistoja, testaavat ja iteroivat, kunnes tuote on valmis julkaistavaksi markkinoille. Tämä prosessi suoritetaan yleensä peräkkäin, ja jokainen vaihe vie huomattavan paljon aikaa. Usein ohjelmistokehitysvaihe kuluttaa eniten suunnittelutehoa ja siitä tulee lopulta suurin pullonkaula.

    Quick-Connect Studion ansiosta insinöörit voivat nyt suorittaa laitteisto- ja ohjelmistokehitystä samanaikaisesti. Tämä on radikaali muutos alalla, jonka avulla suunnittelijat voivat rakentaa ohjelmistoja välittömästi, jotta ne voivat nopeasti määrittää uudelleen ja testata tuoteideoita. Tämä säästää dramaattisesti aikaa ja vähentää riskejä, koska insinöörit voivat validoida suunnitelmat ennen sitoutumistaan laitteistoasetteluun.

    Quick-Connect Studio antaa insinööreille mahdollisuuden rakentaa ratkaisunsa graafisesti vetämällä ja pudottamalla laite- ja alijärjestelmälohkoja pilvessä sijaitsevaan suunnitteluunsa. Kunkin lohkon sijoittamisen jälkeen käyttäjät voivat luoda, kääntää ja rakentaa ohjelmistoja automaattisesti, mikä on merkittävä muutos kohti kooditonta kehitystä.

    Tämä tekee tuotantotason ohjelmistojen rakentamisesta yhtä helppoa kuin rakennuspalikoiden kokoamisen yhteen. Quick-Connect Studion avulla käyttäjät voivat nopeasti rakentaa täydellisen ratkaisun pilveen ja ottaa sen käyttöön laitteistossa alle 10 minuutissa. Vanhaa osaamista tai investointeja tarvitaan hyvin vähän. Pilvilaskentateho mahdollistaa nopean kääntämisen, ja moderni graafinen käyttöliittymä vähentää oppimiskäyrää ja edistää projektin uudelleenkäyttöä.

    Quick-Connect Studio perustuu Quick-Connect IoT:hen, standardisoitujen laitteistojen alustaan, jossa on alan vakiintuneet rajapinnat, kuten PMOD, Arduino ja MIKROE. Vakioliittimillä insinöörit voivat sekoittaa ja yhdistää mikro-ohjaimia, -prosessoreita, antureita ja liitäntäkortteja saumattomasti.

    Renesas esittelee työkaluja Nürnbergin Embedded World messuilla maaliskuun puolivälissä.

    Reply
  25. Tomi Engdahl says:

    Farnellilta nyt tekoälyä verkon reunalle
    https://etn.fi/index.php/new-products/14636-farnellilta-nyt-tekoaelyae-verkon-reunalle

    etson on Nvidian suosittu tekoälyalusta, josta löytyy erilaisia kortteja verkon reunalle suorituskykytarpeen mukaan. Nyt Jetson-kortit ovat tulleet Farnellille, jonka valikoimaan kuuluvat Jetson Nano, Jetson AGX ja Jetson Xavier NX.

    Jokainen moduuli on varustettu Nvidian JetPack-SDK:lla eli ohjelmistonkehitystyökaluilla kiihdytystä varten. JetPack SDK tarjoaa Linux-ympäristön, CUDA-X-kirjastot, API:t ja työkalut tekoälyn reunan kehittämiseen ja tukee myös korkeamman tason kehitysalustoja, kuten DeepStream ja NVIDIA Isaac.

    Reply
  26. Tomi Engdahl says:

    Debugging Raspberry Pi Pico C & C++
    Using Visual Studio Code
    https://www.okdo.com/project/debugging-raspberry-pi-pico-c-c/?ok_ts=1677755618380

    Debugging Raspberry Pi Pico is vital as programs become longer and more complicated, bugs are more likely to be introduced into the code and sometimes they can be quite difficult to track down. Using a debugger will not only help you find the bugs but will also give you a much better understanding of how your code works. This technique also flashes code to the Pico without having to reconnect the USB cable.

    Specialised hardware debuggers are available but for debugging the Pico everything you need is built into the Raspberry Pi 4 Model B or Raspberry Pi 400 when used with the OpenOCD debug translator and Visual Studio Code.

    1. Serial wire debugging

    Raspberry Pi Pico has a special Serial Wire Debug (SWD) connector marked DEBUG on the end of the board, which is used to connect to an external debugger or to the Raspberry Pi GPIO connector.

    Make sure the Raspberry Pi is powered down and the power supply removed before making any connections, then check them carefully before powering on.

    Reply
  27. Tomi Engdahl says:

    Debug .NET apps on Raspberry Pi
    https://learn.microsoft.com/en-us/dotnet/iot/debugging?tabs=self-contained&pivots=vscode

    Debugging .NET apps running on ARM-based IoT devices like Raspberry Pi presents a unique challenge. If desired, you can install Visual Studio Code and the .NET SDK on the device and develop locally. However, the device’s performance is such that coding and debugging locally is not recommended. Additionally, the Visual Studio Code extension for C# is not compatible with 32-bit ARM operating systems. Consequently, functionality like IntelliSense and debugging in Visual Studio Code on ARM devices is only supported in 64-bit systems.

    For these reasons, it’s strongly recommended that you develop your app on a development computer and then deploy the app to the device for remote debugging.

    Debug from Visual Studio Code (cross-platform)

    Debugging .NET on Raspberry Pi from Visual Studio Code requires configuration steps on the Raspberry Pi and in the project’s launch.json file.

    Enable SSH on the Raspberry Pi
    Install the Visual Studio Remote Debugger on the Raspberry Pi
    Set up launch.json in Visual Studio Code
    Deploy the app
    Launch the debugger

    In Visual Studio Code, on the Run and Debug tab, select the configuration you added to launch.json and select Start Debugging. The app launches on the Raspberry Pi. The debugger may be used to set breakpoints, inspect locals, and more.

    Reply
  28. Tomi Engdahl says:

    How to Program Raspberry Pi Pico with Visual Studio Code?
    https://www.mathaelectronics.com/how-to-program-raspberry-pi-pico-with-visual-studio-code/

    This guide will teach you how to use Visual Studio Code to program the Raspberry Pi Pico. An excellent IDE for software development is Visual Studio Code, also known as VS Code. We’ll demonstrate how to install Visual Studio Code on the Raspberry Pi (the host computer), and in a subsequent article, we’ll demonstrate how to do the same for a Windows system (including installation of all the other necessary tools).

    nstalling Visual Studio Code in Raspberry Pi

    IMPORTANT NOTE: I strongly advise you to finish the preceding instructions on the Raspberry Pi Pico before installing Visual Studio Code on your Raspberry Pi host computer. Certainly the instructions for “Programming Raspberry Pi Pico with C” and “Program and Debug Raspberry Pi Pico with SWD.”

    All of the actions that come after presume that you’ve already installed the required toolchains as described in the earlier lessons.

    Open your Raspberry Pi’s browser and navigate to the official VS Code download page first. Download the VS Code ARM.deb package from Linux Downloads by swiping down from the top of the page (64-bit or 32-bit depending on your Raspberry Pi OS).

    Reply
  29. Tomi Engdahl says:

    https://www.raspberrypi.com/news/debugging-embedded-software-with-raspberry-pi-pico/

    The OpenOCD connection

    The debugging itself is managed by the ‘Open On-Chip Debugger’ (OpenOCD) software. This provides a connection between the hardware and the software environment that you use to write and debug your code. OpenOCD talks to the ESP-PROG device over USB. The ESP-PROG provides two serial port connections to the host computer. One can be used for programming an ESP32 via the 6-pin connector on the ESP-PROG. The other is used to control debugging.

    Debugging with Visual Studio Code and PlatformIO

    Visual Studio Code is a free development environment that runs on PC, Mac, and Raspberry Pi. PlatformIO is a free plug-in for embedded development using Visual Studio Code. PlatformIO includes the OpenOCD framework. A PlatformIO project contains a platform.ini file that contains the project configuration options. We need to edit this file and add two lines to our configuration:

    debug_tool = esp-prog
    debug_init_break = tbreak setup

    Now we can open up the debug window in Visual Studio Code and start the debugger.

    Hardware debugging with Raspberry Pi PICO

    The Raspberry Pi Pico device exposes JTAG signals that can be used for hardware debugging. You can wire these directly to a Raspberry Pi and use that as the debugging and development platform, or you can use another Raspberry Pi Pico device as a debugging probe.

    The Pico documentation gives detailed instructions on how to do this here. You can use the GNU Debugger to debug a program from the command line.

    (gdb) b main
    Breakpoint 1 at 0x1000035c: file /home/pi/pico/pico-examples/blink/blink.c, line 9.
    (gdb) continue
    Continuing.
    Thread 1 hit Breakpoint 1, main () at /home/pi/pico/pico-examples/blink/blink.c:9
    9 int main() {
    (gdb) step
    14 gpio_init(LED_PIN);

    The statements above are from a GDB debug session investigating the blink demo program for the Pico. The debugging commands that were entered are shown in bold.

    Hardware debugging for the win

    Hardware debugging is very powerful. It lets you look inside your devices to see exactly what they are doing. You do need to be a bit careful when you use it sometimes, because the debugging process stops the target device and all background processes. On a device like the ESP32, this can cause problems with WiFi and Bluetooth connections being maintained during debugging. However, given the low cost of getting started, you should definitely consider adding the technique to your armoury of tools.

    Reply
  30. Tomi Engdahl says:

    Remote Debugging of Raspberry Pi with JTAG interface
    https://wiki.aalto.fi/download/attachments/84747235/rpi_jtag.pdf?version=3&modificationDate=1386972920322&api=v2

    Many silicon architectures have built-in software support for
    debugging, including ARM CPU of RPi [7]. It is accessed
    through test access ports (TAPs) via a JTAG adapter. A
    TAP might be a module inside one chip or a separate chip
    by itself. To communicate with a JTAG target the host
    connects to JTAG pins by the means of the adapter. The
    adapter takes care of actually forming the necessary signals
    and it is connected to the host computer through USB, Eth-
    ernet, PCI or another interface. Different boards require dif-
    ferent adapters. Other characteristics of an adapter include
    [5]:
    • Throughput
    • Voltage range
    • Supported host software
    In order to utilize the described hardware, the host machine
    should have a special software, i.e. the debugger. The one
    that is used in this work is the Open On-Chip Debugger
    (OpenOCD)

    Finally, the target of debugging is the RPi ARM processor
    [1]. It has a built-in JTAG module that can be controlled
    from the host machine to halt the execution, examine the
    registers state, read and write to the main memory. Over-
    all, the physical layout of the connections, hardware and
    software is presented on Figure 1.

    JTAG is a generic transport interface for debugging circuits
    or using the boundary scan.

    This is the basis of JTAG, and the rest of the functionality
    varies from one chip to another

    RPi board contains GPIO pins that can be configured for dif-
    ferent purpose inputs and outputs. There are several alter-
    native configurations, and one of them enables JTAG func-
    tionality.

    In order to compile the software for RPi it is possible either
    to do it natively on RPi itself, or to use a host machine,
    which is usually a preferable way, because the host computer
    is generally faster than an RPi. However it requires a so-
    called cross-compiling “toolchain” to compile for RPi from a
    machine with x86 processor, for example

    The OpenOCD is an open-source tool for debugging em-
    bedded systems. It runs on the host machine as a server,
    allowing connections over Telnet or from GDB (The GNU
    Project Debugger) [3] in order to debug the target systems,
    connected to the host machine through a JTAG adapter.
    Since there are many variant of the adapters, as well as many
    JTAG implementations, OpenOCD should be configured to
    work with a particular adapter and the board.

    or example,
    the configuration file for Olimex ARM-JTAG-USB adapter
    is the following:
    interface ft2232
    ft2232_device_desc
    “Olimex OpenOCD JTAG ARM-USB-TINY-H”
    ft2232_layout olimex-jtag
    ft2232_vid_pid 0x15ba 0x002a
    It uses FT2232 drivers for the USB interface and configures
    it accordingly. The OpenOCD configuration file for RPi is:
    # Broadcom 2835 on Raspberry Pi
    telnet_port 4444
    gdb_port 5555
    #tcl_port 0
    adapter_khz 1000
    if { [info exists CHIPNAME] } {
    set _CHIPNAME $CHIPNAME
    } else {
    set _CHIPNAME raspi
    }
    reset_config none
    if { [info exists CPU_TAPID ] } {
    set _CPU_TAPID $CPU_TAPID
    } else {
    set _CPU_TAPID 0x07b7617F
    }
    jtag newtap $_CHIPNAME arm
    -irlen 5 -expected-id $_CPU_TAPID
    set _TARGETNAME $_CHIPNAME.arm
    target create $_TARGETNAME
    arm11 -chain-position $_TARGETNAME
    First, it sets up the ports for Telnet and GDB connections.
    Then a new JTAG TAP is created.

    OpenOCD works according to a client/server architecture.
    It allows to configure ports on the host machine, where Tel-
    net or GDB connections are accepted. Telnet session gives
    access to the commands of OpenOCD itself, whereas GDB
    session provides a wider set of capabilities like, for example,
    loading a symbol file.
    The GDB command to enable remote debugging is target
    remote. It should be followed by the address and the port
    number of the machine that is running OpenOCD. If it is
    the same computer that is used as a host for both debug-
    gers and the OpenOCD configuration specified GDB port
    to be 5555, then the command would be target remote
    localhost:5555.
    After GDB is connected to OpenOCD it is necessary to tell
    GDB what symbols are available for the program that is be-
    ing run on the board. The GDB command is symbol-file.
    For example, if the JTAG enabling program was compiled
    with debugging information, then it is possible to load the
    symbols from its executable to GDB and perform typical de-
    bugging tasks, like single stepping, examining memory and
    registers, and so on.
    OpenOCD commands can still be accessed from GDB via
    the monitor command.

    After the hardware is setup and OpenOCD configuration
    files are downloaded, the debug session can start.

    OpenOCD opens the configured ports for Telnet or GDB
    and accepts the connections on those ports. A typical Telnet
    session will start with halt command. After issuing it, the
    processor will be halted, and other OpenOCD commands
    can be given. The execution can be continued with resume
    command, that has an optional argument – the memory
    address, which the execution will be resumed from. Another
    flow control command is step that perform a single stepping.
    OpenOCD comes with a set of commands to introspect the
    status of the processor and memory. Processor registers can
    be displayed with reg command over Telnet. For example,
    register 15 is the program counter (pc). It points to the
    instruction that is to be executed. Modifying it corresponds
    to issuing step command with the parameter – the address
    from which to perform the next step.
    The memory area can be displayed with a set of mdX com-
    mands, where ’X’ is one of ’b’, ’w’, … for a byte, a word and
    so on. The memory can also be written to with a set of mwX
    commands. For example, knowing the RPi memory layout,
    where 0×20200000 is a GPIO base address, (base + 0×28)
    is GPSET0, and (base + 0x1c) is GPCLR0 register; and the
    fact that 0×10000, the 16th bit, corresponds to GPIO LED,
    it is possible to turn the RPi green LED on and off from an
    OpenOCD Telnet session:
    >mww 0×20200028 0×10000 // turns LED on
    >mww 0x2020001c 0×10000 // turns LED off

    When JTAG is enabled and the processor is halted, one
    can upload the program image directly to the memory over
    JTAG. The required OpenOCD command is load_image
    , where the image to be uploaded
    is given as the first parameter, and it will be loaded to the
    address given as the second parameter. GDB has a com-
    mand to load images as well, load. It will automatically
    put the executable to the correct address, specified at the
    compilation time. The uploading speed depends solely on
    the hardware solution used for JTAG. Olimex ARM-USB-
    TINY-H shows the speed of up to 40 KiB/s. So 2 MiB
    file can be uploaded in 50 seconds.

    The speed of writing
    to an SD card is much bigger but a lot of time is wasted
    on physically moving it from an RPi to the host computer
    and backwards. For small images, like bare metal programs,
    downloading over JTAG is definitely faster.

    Kernel debugging is not a trivial
    task, but doing it remotely simplifies the approach a lot.

    By default, JTAG is not enabled in the kernel, so special steps
    are needed before the kernel can be debugged over JTAG.

    The kernel expects to be loaded at the address 0×8000. But
    usually the kernel image is put on the booting partition in
    a compressed form as (zImage) along with the decompres-
    sor. The decompressor is ignorant to where it is loaded to,
    and it is smart enough to uncompress the kernel and put it
    to the correct memory address. This is important because
    when building an image for the RPi that will enable JTAG
    before booting the kernel, the actual zImage will be shifted
    in memory to introduce the JTAG enabling code in front of
    it.

    OpenOCD commands can be issued from GDB with its
    monitor command. For example, monitor halt equals to
    halting the processor from a Telnet session. In the same
    way the disassembled code can be retrieved via monitor
    arm disassemble. All other OpenOCD functions available
    through Telnet protocol work from GDB as well.
    To start booting the kernel it is necessary to manually con-
    tinue the execution from the address where the decompressor
    of the kernel got loaded to.

    Reply
  31. Tomi Engdahl says:

    ACL Access control list
    CMMI Capability maturity model integration
    CMMI-DEV Capability maturity model integration for development
    CMU Carnegie Mellon University
    COTS Commercial off the shelf
    CVSS Common vulnerability scoring syste
    DM Defect management
    e.g. exempli gratia
    FDIS Final Draft International Standard
    HTTP Hypertext transfer protocol
    IACS Industrial automation and control system(s)
    IEC International Electrotechnical Commission
    ISA International Society of Automation
    ISO International Organization for Standardization
    MIN Minimum
    OWASP Open Web Application Security Project
    SL-C Capability Security Level
    SCA Static code analysis
    SD Secure Design
    SDL Security development life-cycle
    SDLA Secure Development Life-Cycle Assessment
    SEI Software Engineering Institute
    SG Security guidelines
    SI Secure implementation
    SM Security management
    SR Security requirements
    STRIDE Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, Elevation of privilege
    SVV Security verification and validation
    TCP Transmission control protocol
    TCP/IP Transmission control protocol/Internet protocol
    TR Technical report
    USB Universal serial bus

    Reply
  32. Tomi Engdahl says:

    PCIe For Hackers: The Diffpair Prelude
    https://hackaday.com/2023/03/14/pcie-for-hackers-the-diffpair-prelude/

    PCIe, also known as PCI-Express, is a highly powerful interface. So let’s see what it takes to hack on something that powerful. PCIe is be a bit intimidating at first, however it is reasonably simple to start building PCIe stuff, and the interface is quite resilient for hobbyist-level technology. There will come a time when we want to use a PCIe chip in our designs, or perhaps, make use of the PCIe connection available on a certain Compute Module, and it’s good to make sure that we’re ready for that.

    PCIe is everywhere now. Every modern computer has a bunch of PCIe devices performing crucial functions, and even iPhones use PCIe internally to connect the CPU with the flash and WiFi chips. You can get all kinds of PCIe devices: Ethernet controllers, high-throughput WiFi cards, graphics, and all the cheap NVMe drives that gladly provide you with heaps of storage when connected over PCIe.

    PCIe is a point-to-point bus that connect two devices together – as opposed to PCI, an older bus, that could connect a chain of devices on your mainboard. One side of a PCIe link is a device, and another is a host. For instance, in a laptop, your CPU will have multiple PCIe ports – some used to connect the GPU, some used to connect a WiFi card, some used for Ethernet, and some used for a NVMe drive.

    Each PCIe link consists of at least three differential pairs – one is a 100 MHz clock, REFCLK, that is (almost) always required for a link, and two pairs that form a PCIe lane – one for transmit and another for receive. This is an x link – you can also have 2x, 4x, 8x and 16x links, with four, eight sixteen and thirty-two differential pairs respectively, plus, again, REFCLK. The wider the link, the higher its throughput!

    Treating Your Diffpairs With Respect

    First off, you want to keep both of the pair’s signals close to each other throughout their length. The closer the two signals are, the better external interference cancellation works, and the less noise they radiate – given that often, multiple diffpairs run next to each other, this will help signal integrity of other pairs as well. Speaking of running separate diffpairs next to each other, you’ll want to keep them away from each other and other things – be it ground fills on the same layer, high-frequency signals. A great rule of thumb is the 5W rule, which says you need to have at least five trace width’s worth of clearance between a diffpair’s trace center and other signals. You don’t always have this much space, but it’s good to adhere to this as much as possible.

    You will also want to make sure that there is an uninterrupted ground path right under these signals, alongside the entire pair – having a ground fill is ideal.

    Then, there’s the little-talked-about matter – impedance matching. If you’re getting a differential pair from point A to B, you will want to make sure that you get the impedance right, and the basics of it are simpler than you might think.

    Now, this means that you have to make sure the impedance for your PCIe link is good along its entire path – which, in practice, means picking suitable connectors and tuning your PCB trace widths and spacings. PCIe hardware is mostly built with 85 Ω impedance in mind. Things like receivers, transmitters, and PCIe-intended connectors are outside your control, and to get the impedance of the entire path is reasonably uniform, you have to adjust the parts under your control to the same value. For a start, if you have to use connectors for your PCIe link, pick ones that don’t have too significant of an impedance mismatch. A good bet is using high-speed connectors or connectors built with PCIe-like signals in mind – full-size PCIe, M.2, mPCIe, USB3, USB-C, and a lot of high-speed connector families from various manufacturers.

    Now to tuning the impedance of your diffpair’s PCB traces. Differential pair impedance depends on a lot of variables in reality, but if you’re a hacker starting out, there are simplified calculators that get you most of the way there – this one is my favourite. Scroll down to “Edge-Coupled Surface Microstrip”, leave track height at 35 for routing diffpairs on 1 oz copper layers, leave dielectric constant at 4.3 unless your PCB fab gives you a different value.

    https://www.multi-circuit-boards.eu/en/pcb-design-aid/impedance-calculation.html

    Now, if you have ever tinkered with PCIe, you might have stumbled upon some forbidden knowledge: in practice, you don’t really-really have to do all of the above.

    You might have heard that PCIe runs over wet string – the first known reference to this is in a 2016 presentation on console hacking at 33C3.

    nd, unsurprisingly, there’s a big grain of truth – PCIe will still work in suboptimal conditions, and there’s an example after example of it in hacker and consumer worlds! Perhaps the most widely available example of PCIe abuse is passing an 1x PCIe link using USB3 cabling, something the “mining” PCIe risers do – which means that you can just go to your computer accessories store and buy a product that is only possible thanks to some PCIe abuse.

    Something else that you might’ve seen and forgotten like a bad dream, is [TobleMiner] putting a x8 PCIe link through, shudder, prototyping wires – for the sake of testing out an adapter idea for cheap high-speed networking cards from HP servers, not compatible with regular PCIe slots both pinout-wise and mechanically.

    PCIe is quite a bit more forgiving than quite a few other interfaces, say, USB3. There are link training mechanisms – when a PCIe connection is established, the receiver and transmitter play around with their internal parameters, adjusting them until they reach the fastest speed possible while keeping error rate low, using these parameters for the entire connection afterwards. There are also retransmissions for packets that failed to be received. PCIe has exceptional stability in practice.

    It’s clear that PCIe link training has some unique parts to it – for instance, to help you make your layout better, PCIe also lets you invert any differential pair, except REFCLK, by swapping the negative and positive signals, and this will be detected and flawlessly compensated for during link training. Other technologies like USB3, HDMI, or DisplayPort don’t support such quality-of-engineer-life features. Other interfaces often require that multiple lanes should be the same length – making sure that data on one set of pairs doesn’t arrive faster than on the other. PCIe, however, is fine with across-pair mismatches as well, also detecting and compensating for these during link training. These two aren’t meant to be resilience features as much as they’re ease-of-layout features meant to help you design PCBs faster and better, but it sure helps that they’re there.

    Try Your Best, No Matter What

    Does this resillience help hackers? Yes, absolutely – these two ease-of-layout features are used in basically any professional PCIe design, and if you’re in less sterile conditions, you can push PCIe further at your own risk. On the other hand, don’t just skirt every rule because you’ve seen someone do that – put some good-faith effort into following these five guidelines, even if you’re limited to a two-layer PCB and might never get the perfect impedance value. Following these rules will not only teach you some diffpair discipline for later projects, it will make your PCIe signals all that more resillient and error-free, and your PCIe devices more happy. It might feel good to dismiss all or some of these guidelines, since sometimes it might just work out, but the extra half hour calculating proper impedance on your board will help you ensure that your PCB doesn’t need a second revision and stays loyal to your interests throughout its entire life.

    So, here’s a guideline: treat your PCIe differential pairs with respect. If you’re using a two-layer PCB and you’re doing a prototype on the cheap and you want quick turnaround time, don’t just give up on impedance because the traces would need to be way too wide to reach 85 ohms – open the calculator and see just how much you can get the impedance down anyway. Lowering isolation height lowers impedance, so consider going for 0.8mm PCB if your project’s mechanical aspects let you. Move your components around if that helps your PCIe tracks follow a better path, with less noise along the way. Perhaps link training will knock an imperfect link down a generation or two, but that’s better than not reaching a stable link at all. Put your best effort following these guidelines with what you’re given, and the differential pairs will respect your intentions in return.

    For instance, if you’re using KiCad, here’s a simple demonstration on how to get a PCIe 1x link from one point to another, routing differential pairs while taking care of impedance, clearances, and via stitching.

    PCIe x1 link diffpair routing example (KiCad 7)
    https://www.youtube.com/watch?v=dZC2e_oUon8

    Reply
  33. Tomi Engdahl says:

    https://www.sparkfun.com/micromod

    What is MicroMod?
    MicroMod is a solderless, modular interface ecosystem that uses the M.2 standard to mix and match your choice of processor with specific Function Boards or stand-alone Carrier Boards.

    Reply
  34. Tomi Engdahl says:

    Extracting Firmware from Embedded Devices (SPI NOR Flash)
    https://www.youtube.com/watch?v=nruUuDalNR0

    One of the first things you have to do when hacking and breaking embedded device security is to obtain the firmware. If you’re lucky, you can download it from the manufacturer’s website or, if you have a shell, you can just copy it over to your computer.

    But what if none of these options are available?

    In this video, we will show you how you can connect directly to a NOR flash chip with the SPI protocol to dump the firmware and find your vulns, even if off the shelf tools don’t work!

    00:00 Intro
    00:40 Technical Introduction
    01:55 Flash Memory Types
    03:51 NOR Flash
    06:25 SPI Protocol
    07:55 Our Training
    09:27 Logic Analyzer
    12:04 How SPI Works
    13:53 Firmware Extraction

    Reply

Leave a Comment

Your email address will not be published. Required fields are marked *

*

*