Embedded linux network device driver development

Embedded Linux Network
Device Driver Development

Rights to Copy
 License: Creative Commons Attribution - Share Alike 3.0
 You are free:
 to copy, distribute, display, and perform the work
 to make derivative works
 to make commercial use of the work
 Under the following conditions:
 Attribution – Derived from original work of Free Electrons
 Share Alike
 For any reuse or distribution, you must make clear to others the license terms
of this work.
September 7, 2017
Embedded Linux Network Device Driver
Development
2

References
 www.free-electrons.com
 Essential Linux Device Drivers
 Linux Device Drivers
September 7, 2017
Development
3

Objectives
 Understanding the structure of ethernet and WIFI device drivers
 Developing Linux network device drivers
Development
4 September 7, 2017

Prerequisites
 Solid C programming
 Knowledge of Linux commands is a plus
 Knowledge of networking
 Embedded Linux Kernel and Driver Development training or equivalent
Development
5 September 7, 2017

Notes
 Ask any time.
 Turn your cell silent.
Development
6 September 7, 2017

Outline
 Introduction
 Socket Buffers
 Network Devices
 Communication with Network Protocol
 Communication with PHY
 Buffer Management and Concurrency Issues
 Network Throughput
 Other Network Driver Considerations
 Ethernet Driver vs WIFI Driver
September 7, 2017
Development
7

Outline
 Introduction
 Socket Buffers
 Network Devices
September 7, 2017
Development
8

OSI Model and Linux Kernel
September 7, 2017
Development
9
User Space
Kernel Network
Stack
Device Driver

Network Device Model
September 7, 2017
Development
10

Outline
 Introduction
 Socket Buffers
 Network Devices
September 7, 2017
Development
11

Socket Buffer (sk_buff)
 struct sk_buff (include/linux/sk_buff.h) represents a network packet
 Support data encapsulation/decapsulation through protocol layers
 In addition to data, sk_buff maintains:
 head, the start of the packet
 data, the start of the packet payload
 tail, the end of the packet payload
 end, the end of the packet
 len, the amount of data of the packet
 These fields are updated when the packet goes through the protocol
layers
September 7, 2017
Development
12

SKB Operations
Allocation
 By dev_alloc_skb() function
 Can be done in ISR
 On Ethernet, allocated size = packet
length + 2
 To word align IP header
 IP header = 14 bytes
Reservation
 By skb_reserve() function
 Skipping NET_IP_ALIGN padding
bytes
September 7, 2017
Development
13

SKB Operations cont’d
Data Copying
 From DMA buffer to SKB during
reception for example
 OR SKB to DMA buffer during
transmission (may be)
Pointers Update
 After copying payload
September 7, 2017
Development
14

Outline
 Introduction
 Socket Buffers
 Network Devices
September 7, 2017
Development
15

Network Device
 struct net_device (include/linux/netdevice.h) represents a network
interface
 Allocation takes place with alloc_netdev() function
 Size of private data is passed as argument
 Pointer to these private data can be read in net_device->priv
 Use alloc_etherdev() for ethernet interfaces
 Use alloc_ieee80211() for WIFI interfaces
September 7, 2017
Development
16

Network Device cont’d
 Registration with register_netdev() function
 Unregistration with unregister_netdev() function
 Liberation with free_netdev() function
September 7, 2017
Development
17

Network Device Operations
 Defined by struct net_device_ops (include/linux/netdevice.h)
 Set the netdev_ops field in the struct net_device structure to point to the
struct net_device_ops structure
September 7, 2017
Development
18
Operation Description
ndo_open() Called when network interface uped
ndo_close() Called when network interface downed
ndo_start_xmit() Start packet transmission
ndo_get_stats() Get statistics
ndo_do_ioctl() Implement device specific operations
ndo_set_rx_mode() Select promiscuous, multicast, etc
ndo_set_mac_address() Set MAC address
ndo_set_multicast_list() Set multicast filters

Network Device Operations cont’d
 Sample operations are:
September 7, 2017
Development
19
ndo_open() Called when network interface uped
ndo_close() Called when network interface downed
ndo_start_xmit() Start packet transmission
ndo_get_stats() Get statistics
ndo_do_ioctl() Implement device specific operations
ndo_set_rx_mode() Select promiscuous, multicast, etc
ndo_set_mac_address() Set MAC address
ndo_set_multicast_list() Set multicast filters
ndo_tx_timeout() Reset unresponsive network interface

Outline
 Introduction
 Socket Buffers
 Network Devices
September 7, 2017
Development
20

Flow Control Utility Functions
 include/linux/netdevice.h
September 7, 2017
Development
21
netif_start_queue() Tells kernel that driver is ready to send packets
netif_stop_queue() Tells kernel to stop sending packets (@cleanup or congestion)
netif_queue_stopped() Tells whether queue is stopped or not
netif_wake_queue() Wakes up a queue after a netif_stop_queue()
netif_rx() Gives SKB to kernel

Transmission
 Kernel calls ndo_start_xmit() with SKB as argument
 ndo_start_xmit():
1. Sets up DMA buffers and other HW mechanisms
2. Starts transmission
3. Can stop queueing if no more free DMA buffers available using
netif_stop_queue()
4. Returns NETDEV_TX_OK or NETDEV_TX_BUSY
 When N packets have been sent, an interrupt is raised and the driver
should:
1. Acknowledging interrupt
2. Freeing used DMA buffers
3. Freeing SKB with dev_kfree_skb_irq()
4. If the queue was stopped, start it again
September 7, 2017
Development
22

Reception: Non-NAPI
 Reception is via an interrupt that should:
1. Allocate an SKB
2. Reserve NET_IP_ALIGN padding bytes
3. Copy packet from DMA buffers to SKB
4. Update SKB pointers
5. Update the skb->protocol field with eth_type_trans(skb, netdevice)
6. Give SKB to kernel by netif_rx()
 netif_rx() use NET_RX_SOFTIRQ to offload the work of posting received data
packets to protocol layers
 Nice and simple
 @high traffic, interrupt rate is high
 Solution: switch to polled mode when interrupt rate is too high (NAPI)
September 7, 2017
Development
23

Reception: NAPI
 In network interface private structure, add a struct napi_struct
(include/linux/netdevice.h)
 @ driver initialization, register the NAPI poll operation
 dev: network interface
 &lp->napi: struct napi_struct
 r6040_poll: NAPI poll operation
 64: weight that represents the importance of network interface
September 7, 2017
Development
24

Reception: NAPI cont’d
Rx ISR
 Disables Rx interrupt and switch to
polled mode
if (napi_schedule_prep(&lp->napi)) {
/* Disable reception interrupts */
__napi_schedule(&lp->napi);
}
 Kernel calls our poll() operation
regularly
Poll Function
 static int r6040_poll(struct
napi_struct *napi, int budget)
 Receives at most budget packets and
pushes them to the network stack
using netif_receive_skb()
 If less than budget packets have been
received, switch back to interrupt
mode using napi_complete(&lp-
>napi) and reenable interrupts
 Must return the number of packets
received
September 7, 2017
Development
25

Outline
 Introduction
 Socket Buffers
 Network Devices
September 7, 2017
Development
26

Connection with PHY
 MAC and PHY are connected using a MII or RMII interface
 This interface contains two wires used for the MDIO bus
 Driver needs to communicate with the PHY to get link information (up,
down, speed, full or half duplex) and configure the MAC accordingly
September 7, 2017
Development
27

PHY in Kernel
 Kernel has a framework (drivers/net/phy/) that
 Exposes an API to communicate with PHY
 Allows to implement PHY drivers
 Implements a basic generic PHY driver that works with all PHY
 See Documentation/networking/phy.txt
September 7, 2017
Development
28

MDIO Bus Initialization
 Driver creates a MDIO bus struct mii_bus (include/linux/mii.h) to tells PHY
infrastructure how to communicate with the PHY
 mdio_read() and mdio_write are HW specific and must be implemented
by the driver
September 7, 2017
Development
29

MDIO Bus Initialization cont’d
 The ->irq[] array must be allocated and initialized
 To use polling, set the values to PHY_POLL
Lp->mii_if->irq = kmalloc(sizeof(int)*PHY_MAX_ADDR, GFP_KERNEL);
for (i = 0; i < PHY_MAX_ADDR; i++)
lp->mii_if>irq[i] = PHY_POLL;
 Finally, register the MDIO bus
 This will scan the bus for PHYs and fill the mii_if->phy_map[] array with the
result
mdiobus_register(bp->mii_if);
September 7, 2017
Development
30

Connecting with PHY
 mdiobus_register() function filled the mii_if->phy_map[] array with struct
phy_device * pointers
 Appropriate PHY (usually, only one is detected) must be selected
 Connecting to the PHY allows to register a callback that will be called
when the link changes :
 interface is usually PHY_INTERFACE_MODE_MII or
PHY_INTERFACE_MODE_RMII
September 7, 2017
Development
31

Updating MAC Capabilities
 MAC and PHY might have different capabilities
 PHY handling Gigabit speed, but not MAC
 Driver is responsible for updating phydev->advertise and phydev-
>supported to remove any PHY capability that the MAC doesn't support
September 7, 2017
Development
32

Handling Link Changes
 The callback that handle link changes should have the following prototype:
void foo_handle_link_change(struct net_device *dev)
 It must check the duplex, speed and link fields of the struct phy_device
structure, and update Ethernet controller configuration accordingly
 duplex is either DUPLEX_HALF or DUPLEX_FULL
 speed is either SPEED_10, SPEED_100, SPEED_1000, SPEED_2500 or
SPEED_10000
 link is a Boolean
September 7, 2017
Development
33

Starting and stopping the PHY
 After set up, to make the PHY driver poll regularly the PHY hardware, one
must start it with phy_start() functions
 And stop it using phy_stop(), when no longer needed
September 7, 2017
Development
34

Outline
 Introduction
 Socket Buffers
 Network Devices
September 7, 2017
Development
35

Buffers Management
 During open(), driver pre-allocates needed DMA descriptos needed for
transmission and reception
 When totally utilized, the driver tells the kernel to stop sending packets to
the driver
 SKB are allocated and freed as per need
September 7, 2017
Development
36

Concurrency
 Access protection needed in the face of multiple execution threads:
 Transmit thread
 Receive thread
 Transmit-complete interrupt
 Receive interrupt
 NAPI polling
September 7, 2017
Development
37

Outline
 Introduction
 Socket Buffers
 Network Devices
September 7, 2017
Development
38

Network Benchmarking
 Netperf (free from www.netperf.org ) can set up complex TCP/UDP
connection scenarios to control C/C’s such as protocol parameters,
number of simultaneous sessions, and size of data blocks.
 Benchmarking = Comparing resulting throughput with maximum practical
bandwidth
 Factors affecting throughput:
 Driver implementation
 Protocol used
September 7, 2017
Development
39

Driver Performance
 Execution time
 Considering usage of DMA for large packets
 Offloading CPU by moving functionalities to hardware (TCP checksum)
 Taking into consideration upper layers behavior
September 7, 2017
Development
40

Protocol Performance
 TCP window size = data that can be transmitted before receiving ACK
 Fast network interfaces + small window size = TCP sitting idle waiting for ACK
 In UDP, the window size is not relevant as there is no ACK
 Packet loss in TCP and UDP
 Mapping protocol block sizes to MTU sizes
September 7, 2017
Development
41

Outline
 Introduction
 Socket Buffers
 Network Devices
September 7, 2017
Development
42

Ethtool (include/linux/ethtool.h)
 ethtool is a user space tool that query low-level information from an
Ethernet interface and to modify its configuration
 @ driver, a struct ethtool_ops structure can be declared and connected to
the struct net_device using the ethtool_ops field
 List of operations: get_settings(), set_settings(), get_drvinfo(), get_wol(),
set_wol(), get_link(), get_eeprom(), set_eeprom(), get_tso(), set_tso(),
get_flags(), set_flags(), etc.
 Some of these operations can be implemented using the PHY interface
(phy_ethtool_gset(), phy_ethtool_sset()) or using generic
operations(ethtool_op_get_link() for example)
September 7, 2017
Development
43

Statistics
 The driver is responsible for keeping statistics up to date about the
number of packets/bytes received/transmitted, the number of errors, of
collisions, etc.
 To expose these information, the driver must implement a get_stats()
operation, with the following prototype struct net_device_stats
*foo_get_stats(struct net_device *dev);
 The net_device_stats structure must be filled by the driver
 It contains fields such as rx_packets, tx_packets, rx_bytes, tx_bytes, rx_errors,
tx_errors, rx_dropped, tx_dropped, multicast, collisions, etc
September 7, 2017
Development
44

Power Management
 suspend() function should:
 Call netif_device_detach()
 Do the hardware dependent operations to suspend the devices (like disable
the clocks)
 The resume() function should:
 Do the hardware dependent operations (like enable the clocks)
 Call netif_device_attach()
September 7, 2017
Development
45

Lab: Training Setup
September 7, 2017
Development
46

Outline
 Introduction
 Socket Buffers
 Network Devices
September 7, 2017
Development
47

Ethernet vs. WIFI
Ethernet
 CSMA/CD
 Frames are not acknowledged
WIFI
 CSMA/CA
 Frames are acknowledged
 Uses WEP for enhanced security
 2 modes of operation:
 Ad-hoc
 Infrastructure
September 7, 2017
Development
48

Configuring WIFI Drivers
 The Wireless Extensions project defines a generic Linux API to configure
WLAN device drivers in a device independent manner
 It also provides a set of common tools to set and access information from
WLAN drivers
 Individual drivers must implement support for Wireless Extensions to
connect themselves with the common interface and, hence, with the tools
September 7, 2017
Development
49

Talking to WIFI Drivers
 Using iwconfig utility
 To glue your driver to iwconfig, you need to implement prescribed functions
corresponding to commands that set parameters such as ESSID and WEP keys
 Using iwpriv utility
 To use iwpriv over your driver, define private ioctls relevant to your hardware
and implement the corresponding handler functions
 Using /proc/net/wireless to get driver statistics
 For this, implement the get_wireless_stats() function in your driver, in addition
to the get_stats() function
 WLAN drivers tie these three pieces of information inside a structure
called iw_handler_def, defined in include/net/iw_handler.h
September 7, 2017
Development
50

Ethernet/WIFI Famous Device Drivers
September 7, 2017
Development
51
Device(s) Driver Location in Kernel Tree
Intersil Prism2 WLAN Compact Flash Card /drivers/net/wireless/intersil/orinoco
Intel Pro/Wireless Mini PCI (and PCIe Mini) /drivers/net/wireless/intel/ipw2x00
Atmel WLAN USB /drivers/net/wireless/atmel
Intel PRO/1000 /drivers/net/ethernet/intel/e1000

Ethernet Driver Example
September 7, 2017
Development
52

Lab: Ethernet Driver
September 7, 2017
Development
53

WIFI Driver Example
September 7, 2017
Development
54

Lab: WIFI Driver
September 7, 2017
Development
55

 To contact us:
 https://www.facebook.com/groups/EmbeddedSystemsTraining/
 www.swift-act.com
 training@swift-act.com
 (+2)0122-3600-207
September 7, 2017
Development
56

Embedded linux network device driver development

More Related Content

What's hot (20)

Similar to Embedded linux network device driver development (20)

More from Amr Ali (ISTQB CTAL Full, CSM, ITIL Foundation) (20)

Recently uploaded (20)

Embedded linux network device driver development