Lesson 1

Features and constraints in embedded system

1.1. What is embedded system?

Embedded systems, the term, usually refers a lot to, not just how the device is used, but
also how it's implemented, how it's built, so there is a lot of overlap between the two
terms. And IoT devices are typically embedded systems also, so we'll just define that and
give you an idea of what embedded systems are. It's good to know the term embedded
system because it is still commonly used, and what we are building are typically
embedded systems. We will talk about building later in the class. They are usually
embedded systems.

So, what are embedded systems? Embedded systems are computer-based systems that
don't look like computers. That's my simple definition. The complexity of a computer is
hidden from the user. So, if you have a computer, desktop laptop something like that. If
you've ever used it, which I know you have, there are complexities to using a
computer. So, say you want to install some new software. Right? You install
software. Sometimes it's easy. But sometimes there's a conflict, like for instance video
games. I used to play video games on computers, now I use only game machines, but say I
play a video game on a computer. I put the video game in there and install it, and it has
some conflict with the video card. I need a new video card, right, to play this new video
game.

 So, then I need to get new drivers for my video card, so there's this whole interaction. The
functions are not separated. This is different than an IOT device. An IOT device basically
has one function. We talked about this last module, right. It has one function it's trying to
do, like the car. It does the car things, right. A camera does camera things, but a general-
purpose computer It can do a lot of things, and so there can be conflicts. If I install
software to play a video game, that might interfere with the software to do something
else. So, these relationships add to complexity. So, makes it harder to use a standard
computer than to use, say IoT device.

Now an embedded system, and IOT devices are generally embedded, they hide the
complexity from the user. So, the picture we've got here this is like some kind of a scale
let's say, and say it's an IOT scale, so the user doesn't have to see the complexity of what's
going on inside. The user just knows how to use it, so it has a very simple interface. And
this is where the term embedded comes from. The complexity is embedded inside the
device, the user doesn't have to deal with the complexity. So that's what an embedded
system is, how the term came about. Now these embedded systems are much more
common in desktops and laptops because there is computational ability built into a lot of
different devices. Now one thing to notice about embedded systems isn't necessarily
different. Not necessarily but can be different than IOT devices. IOT devices are almost
always connected to the internet, thus the term Internet of Things. Embedded systems
may or may not be. Embedded systems may have computational complexity in there,
but no network connection at all. And so,ss there are a lot of uses for devices like
that. Things are moving from embedded to IOT because of the trends we talked about in
the last module, the fact that internet connectivity is so ubiquitous and it's cheap to
obtain. So, a lot of these devices that used to be just embedded, are now becoming
networked and so you'd call them IOT.

So, what are embedded systems? They're basically everything that interacts, all the
internet of things devices that we've talked about. But plus, or minus the internet part. So,
take a digital camera, that's what we have here. Digital camera, a basic digital camera
might not be networked, right? It might just be an embedded system where you click,
press a button, you take a picture. And the picture is stored locally, but it might not be
immediately networked, right? So that would, you would call that type of thing an
embedded system. Because the complexity is embedded within the device. So, a camera,
an old camera, you know? A mechanical camera has the same interface. You press a
button, it takes a picture, right? New digital camera, same interface, press a button, takes
a picture, but there's a lot more complexity to what's going on inside. So, that's the term
embedded, right? The complexity is embedded inside the device, and the user doesn't
have to see it. The user can benefit from the complexity but using a very simple interface.

Now sometimes embedded systems don't interact directly with the user, but they interact
with the user through another device. So, what do I mean by this? Let's take this, take this
memory, this thumb drive right. Now a thumb drive, you've seen this type of thumb drive,
it stores data. This type of thing does not interact directly with the user, right. A human
doesn't have, doesn't connect this up to their body or something, right, yet. Right now,
they connect it to their computer or something like that or their phone or whatever it
is. And then they interact with it, they can access the files on it through another device. So,
you would also call this an embedded system, even though it's not directly interacting
with the human.

And you see this in other devices like a car. The anti-lock braking system inside a
car. That's interacting with a human but not directly, right? The human presses the brake
pedal, and the brake pedal talks to the anti-lock braking system, so the human interacts
with the car, and the car talks to its sub-system. One big property of embedded systems
and a difference in embedded system design is that efficiency is important. So, what that
generally means is, that It's not enough to get the design to work, to do its task. It must do
its task in an elegant, let's say an elegant way, okay? Either it must do it fast, or it must do
it with low power, or it must do it at low price, right? So, it's not enough just to get it to
work. And this is actually a big difference between embedded system design and,
say, traditional software design. So, I teach programming too.

 When we teach programming, generally, we just teach how you get something to
work. Here, your goal is to get this code to do what it's supposed to do. Whatever I want to
do, sort this list of items, or something like that, all right? But I'm not usually saying in that
class, it must sort this list of items, and it must use the minimum amount of memory
possible. I'm not saying it must sort these items, and it must do it within one
second. Right? I don't put these constraints on it. I just say get it to work. But with
embedded system design, you care a lot about the efficiency. You can't just get it to
work. It's got to do it in an efficient manner. So, what that means, is that, so that the
reason for these constraints is that, you know, most of these devices are used in cost
critical markets. Or in life critical, right? So, if it's using medical or military, people's lives
depend on these devices. And if using cost-critical device that compute consumer
electronics, the cost is important. But either way there's some constraints that are
important to you. Besides just getting the thing to work, it's got to work within a tight set
of constraints. So, for instance, say it's a consumer device like a phone. So, manufacturing
costs, and design costs, and time-to-market, those things are primary. Because you want
it to be cheap, right? You want to make money off the device, so the cost must be
low. Also, time-to-market because you have competitors. And this competitor's coming
out with a new phone on this date, so you must come out on the same date with a phone
that's better than them, let's say. So those things are primary.

But on the other hand, if you're, this is the military. This is going into a tank or weapon
system, and people's lives depend on it, and its life critical. Then, you care about
performance and power more. So, if you're in the military, you might say, look, we'll pay
millions of dollars for this device if it works, and it can effectively do what we need it to
do. So, cost becomes less important. But power and performance really matter. And same
thing with medical, right? You might be a lot more willing to pay a lot of money for a
device. But its better work and it better, you know, maintain your heart the way that it's
supposed to. Like a pacemaker. That thing better be, have high reliability and it better,
you know, rate your, control the pace of your heart properly. So, if that means costing
more money then so be it. So, in these imbedded devices, there's always this tight set of
constraints that you're working under. So, it's not enough to get it to work, you got to get
it to work within the set of constraints that you're given.

So, this is very different than traditional software engineering, where you just want to get
it to work and get it to work somehow. And the assumption there is that Moore's Law will
save you eventually so by Moore's Law, you may have heard of this before, but Moore's
law just says let's say that approximately says that machines get faster and more
computationally efficient every year, right? They get denser and faster. Which by the way,
Moore's law is slowing down but still, this happens? Machines improve. So, software
engineering often you'll just either say, look, just get it to work and if it's slow it's okay,
because process or performance is improving every year. And so, in the future it won't
matter. Right. So that was the background assumption to a lot of software
engineering. But a better system, you can't think like that. You must think look not only
does it work. It's got to be working fast. It's got to be working low power and all this.

1.2. More on Embedded Systems

Embedded devices are generally application specific. Unlike desktop, laptop computers
which can run any type of program or pretty good doing any type of program. Embedded
systems are made to do one thing or one related set of things. And we already talked
about this with IoT devices. So, the phone sort of blurs that line a little bit, all
right, because a phone can do a heck of a lot. You can run arbitrary apps on a phone, but
most devices like the camera in front of me, that thing is made to do camera, right? To
record, play back, something like that. And it does all things related to camera
activities. So, that's common. That's typically how embedded systems are. And the reason
why this is important is because it changes the way you design a device. When you know
that the device is going to do one thing, you design it for the one thing, you don't design it
for everything.

So, the design is focused on the one application, unlike general purpose systems like a
laptop, desktop. So those things essentially end up being overpowered for whatever you
must do because they must be ready to handle any type of computational task. So, for
instance take a standard machine, say a quad-core, four processors they have, quad-cores
not uncommon. So, say you get a quad-core i7, something like this, it's running at 4 GHz,
right? Generally, there is no way that regular users are using up all the
computational potential of that machine. There is no reason why you need four
processors to run Word or PowerPoint, or whatever application you're running. You don't
need 4 GHz to run these applications. So, 99.9% of the time, these laptops and desktops
are well underutilized. Only 1% of the CPU, some tiny fractions of the CPU are being
used. But then occasionally, you need them to do something. So, say, most of the time,
like say, cameraman right here, okay. He's using a, he's got a Mac. I bet you he uses
Photoshop or other kind of tools to do video processing, right. Now, when he does that,
then it's using all the processing power, right, because these video operations chain up
effects on screen images. Those require a lot of computational complexity. But 99.9% of
the time, when this thing is just running PowerPoint, something like that, that's not using
anything. But occasionally, it's running an application that really uses it. So, in some
sense, general-purpose processors, most of the time, they are wasted. There's no reason
to have all that power except in very rare cases. Maybe I'm playing a video game. Video
games are another case where you really use a computational power because they're
doing video operations all the time. 

Graphics, complicated graphics. So, when I'm playing in my video game, okay, I need the
processing power, but most of the time I don't need it. So, in some sense, general purpose
machines are very inefficient in that way, and because of that they cost more than they
need to, to do any individual task. So, what I mean by that is say you want to play video
games, and you want to play the newest, best video games. You can buy a computer, a
good computer with a great graphics card and all this, and that might cost you $1,500,
right? To get the newest, most up-to-date video card, in the best computer, to run the best
games, and run them fast. Or you could say, look, let me just buy an Xbox One, or
PlayStation 4, which is a lot cheaper, say $500. But it runs that game better, probably
better than the best, than that more expensive computer, right? Now, all it does is games,
but it does games great. Right? It does game the best. So, in this way, the design is much
more efficient. It's being used to its full potential because all you ever do with the Xbox 1 is
play games with that thing. And most of the time, they're video. So, you're using the
capacity of that thing, basically all the time you're using it, you're using it to, if not full, but
at high capacity. As opposed to a general-purpose machine. So, this application
specifically lets you change the way you do design, so that you can just put in the things
that you need to put in. Spend money on what you need to put in. So, higher design
efficiency is possible. So, what that means is that, if I know that I'm playing video
games, there are certain pieces of hardware that I don't need in my design, right? Why buy
that, right? If I know that, take this camera, right, this camera in front of me, I know that
this thing is doing some types of video processing. I know it needs to control a lens and
stuff like this, right? But it's never going to have to do other types of processing. I'm not
going to be doing sorts, or something, right? So, I can change the way I do design to just
include the software and the hardware that I need for the task that it's made for. And so
that makes it cheaper. Now it makes it more limited. It can only do what it's made to do,
but what it does, it does cheaply and efficiently.

Another big difference in the way you design embedded systems as compared to sort of a
regular desktop laptop, is that hardware and software are usually designed together. So
again, this is for efficiency reasons. But if, say you want to buy Microsoft Word or
something, some typical desktop software. That Microsoft Word, you buy that separate
from when you buy your computer. Well, maybe you buy your computer, and it comes
with Word but, you can buy your computer and then you can buy Word completely
separately, okay? And there are two different companies. Maybe I bought my computer
from Dell, and I bought Word from Microsoft. Two different companies. One does the
hardware; one does the software. Now, more and more, these companies are coming
together. And sort of Apple is the epitome of this, right? They make their devices'
hardware and software together. They make the phone and the software or whatever the
device is in the software, as one.

Which means that they work together better, okay? And in fact, the Apple machines
traditionally are very reliable because of that. But it also makes them more efficient. So,
you can say, look, since this is just a phone, I don't need to be able to drive servos or
something, right? I can make my hardware just the right hardware to run exactly the
software that I need, right? You can match them. Your whole design process gets more
efficient. And we are going to see this later in the class when we build these things. We are
only going to buy the components we need to build the system we want. We're not going
to say, every system. I'm going to put all this hardware and software together. No, we're
going to buy, we say we need this component. We need an LED. 

We need this, we need that. We hook them up and we make this one system that we want
and write code that uses only that. As opposed to a general-purpose system where you
must have hardware for everything and software for everything just for the one occasion
where you need it. So, you're designing the hardware and software together, and that
makes design a lot harder. Because you must not only be a hardware designer, but you
also must be a software designer. So, when you're writing the software, you must think
about what the hardware can do. See, normally when you take an application like
Word, that thing runs on many different hardware platforms. You generally as a
programmer do not have to be aware of this hardware platform that it's running on. You
just say look, as long it's running on Windows, I don't care, right? 

It doesn't even have to run on Windows. It can run on many applications, and you don't
care about that as a programmer. But if you're writing code for an embedded device, like a
cell phone, you must know exactly what the hardware can do. You must know; I have a
screen. It has this size, right? So, I'm limited in how I write my software to use up that area,
right? I have touch senses, which can sense this and that, right? So, I got to place my
buttons so that everything can be touched. So, you must really be aware of the hardware
when you're designing the software and vice versa, which changes the design process and
makes it frankly, more difficult. Yeah, so more work for designers because you must
understand some of both.

1.3. Generic Embedded Systems Structure

So, here's my high-level generic view of what an embedded system looks like. So, if you
look at an embedded system, it's got to take data from the outside world and then it's got
to output data to the outside world. So, it has, first for the receiving data from the outside
world, it has a set of sensors. So, these sensors they receive information from the
outside in many ways because there are lots of different types of sensors. The most basic
type of sensor would be a button, push button, something like that. And that's receiving
data that somebody types of in. You know, zero, one is binary. Then there's also, for
instance, you could receive sound information like the microphone in front of me or
connected to here. And that receives audio information. There's also video, like the
camera in front of me, or just a regular light sensor that receives video information. There
are all kinds of sensors. Touch screens, right? You can receive touch information through a
touch screen. So, there's a lot of different sensors that an embedded system 
can receive input from. 

So that input comes into a set of sensors and it's going to go into the core of the system
and in the end, once the system has decided, process that information, decide what to do
with that information, it needs to cause some results. An effect to happen in the outside
world. 
So that's done through the actuators at the other side, the actuators, they cause events to
happen in the world. So those might be something like an LED, a light, like there's a light
flashing in front of me. That LED is an output actuator. And it goes off and on to indicate
probably that it's recording, okay? But there are other actuators too, like for instance, the
camera in front of me. It's got motors inside that control the lens to control the focus. So
those motors, those are actuators. Those are outputs of the system. So, the system will
drive those motors and those will be considered actuators and make them.
Pull the lens in and out and things like this. So, lots of different types of
actuators. Speakers are actuators, they output sound. Lights are actuators, they output
light. The screen is an actuator, it also outputs light but in a more interesting, I don't know
if you call it interesting, but in a more detailed format. More complex format a light, so a
screen is an output. So, there are lots of different actuators. So, if you look at the
embedded system, it's in between there, right? It receives data from the sensors, it does
something with the data, and then its outputs to the actuators to make something happen
in the real world because of the data that it received. So, the interface components of the
sensors and the actuators. Now in the middle of the system, the centre of the whole thing
is that microcontroller in the centre. Now, the microcontroller we will talk about in the
next lecture in more detail, and a lot of this class is specialization focus on
microcontrollers and how to program them, but there are other components that are
commonly inside the core of an embedded system as well, and you see two of them right
there. IP and FPGA, we'll talk about those in a little bit right now.

So, IP stands for intellectual property, or IP Core people call them, and intellectual
property core is basically a predesigned component, typically a premanufactured
component so an integrated circuit, a chip that performs one function. When I say one
function, maybe a set of functions, but all related to one task. So, it's a chip that is not
general purpose programmable. It's a core that just performs some small set of functions
that are all related.

So these things can be very useful. They're cheap in high volume. So, what that means is,
high volume means manufacturing volume. So, if you have some very common tasks
that's performed by a lot of systems, like say you got a cell-phone, right? And then that cell
phone, let's say, it's very common to perform some type of audio filtering task, okay? So,
all our cell phones have to do this audio filtering, right, let's say. So, you might make one
of these IP cores, which performs filtering, and then it would be high volume, because
there are many millions of cell phones that are for sale, and so you manufacture many of
them. So, there's a lot of work to designing one of them and it costs a lot if you want to
just fabricate one. But when you're fabricating a high volume, it's very cheap. So, these IP
cores can be very valuable when you have high volume production.

So they're useful for common tasks, that's the best thing. To make sure you get high
volume. You basically want to have; you want to use them for very common tasks. Tasks
that happen over and over because there's a lot of need. So, if it's just a one-off if this is a
design, function that's only done in this one system, then you would almost certainly not
make an IP core and make a special purpose chip just for that. You wouldn't do that, but if
it's some common tasks that's done a lot. So, for instance, network controllers,
right? That's in a lot of systems. Basically, every Internet linked device has some sort of
network controller in it. So that type of thing you might implement as an IP core, a
dedicated piece of hardware which just does IP control operations. 
Audio, video codecs right, so I'm coding audio and video, and compressing it,
decompressing it. These are operations that perform by lots and lots of IOT devices,
so this type of device, this type of function you might put into an IP core. Now these IP
cores, we're not designing them in this class, that's outside of the scope of the class, but
when you're making an IOT System, you might want to buy one of these. So, you'll look at
a library, an online library, of IP cores, find some vendor, Texas Instruments, say, they sell
a lot of chips that do different things, and you find the one that does exactly what you
want. You say, well I'm doing MPEG encoding let me find the TIM peg encoding chip,
and you find, you'll probably find 50 of them, and you pick the one that you want, and
that's the core that you use. 
So that's how we use these things. We're buying them off the shelf, we're not actually
going to be designing IP cores, because that's too complex. And they must interact with a
microcontroller. So, remember that the microcontroller is the centre of all the action,
right? So, it's connected to the IP cores, and it tells them when to do what they need to
do. Right, so if an IP core is for encoding video, the microcontroller will say encode now,
right, encode this data now. And when I say it will say that, it means, I mean through the
signals, so the wires It will send the appropriate sequence of input signals to trigger it to
record. And then the results may be the core sets a signal high when it's completed. And
that goes back to the microcontroller. The microcontroller knows okay the coding is done,
let me move on right. So, there's this conversation between the micro controller and the
IP core that goes on. And so, you have to wire it appropriately and we'll only touch on that.

Now another component that you often find in IOT devices, embedded devices in general,
are Field Programmable Gate Arrays. Now, we will not be using Field Programmable Gate
Arrays in this class because their complexity is basically outside of the scope of the
class, but what they are, they're an interesting thing and you should know what they
are. They're hardware. They're integrated circuits, they're chips. Not programmable chips,
they're regular chips. Not programmable in the conventional sense, okay? These things
can be reconfigured, essentially rewired to perform different tasks. So, let's say you need
something to do audio filtering. So, you might configure the FPGA to perform filtering. So
it's a hardware device that will perform filtering because it's been wired to do that. But
then you can rewire it, reconfigure the device, to rewire it so that it performs mpeg coding
instead or something like that. So, you can rewire this thing, reconfigure it, so that it can
perform different tasks. Now it's not programmable in the sense of a general-purpose
processor, you don't write code for it.

And as a result, it's a lot faster than a general-purpose processor right. You can get that
same flexibility with a general-purpose processor. You could run one program, or another
program, you could do that with a processor, but you would do it with the FPGA if possible
because it's a lot faster. It will execute a lot faster. Because the software conversion step
we won't talk here that is skipped if you directly in hardware using FPGA, so FPGAs have
that advantage. And no fabrication is needed. So, when you make an IP core, and you
fabricate it as a chip. You must fabricate it. You must manufacture chips and that's
expensive. So, it only makes sense in high volume with FPGAs you don't have to fabricate
you just reconfigure it through programming, just connect it by USB to a computer
and you can rewire the thing. So that's much more convenient and cheaper, so if you have
a one-off design you would implement in an FPGA over an IP core.