BUD17-309: IRQ prediction
4 likes•732 views
The document discusses the optimization of CPU idle states through the cpuidle framework, emphasizing the importance of predicting system events to minimize performance impact. It outlines various sources of wakeup interrupts, including inter-processor, timer, and I/O interrupts, and examines current challenges in accurately predicting these events. The text proposes methodologies for analyzing interrupts to enhance prediction accuracy, while highlighting the necessity of new tools and potential improvements to scheduler behavior for energy efficiency.
BUD17-309: IRQ prediction
- 1. IRQ next prediction Daniel Lezcano
- 2. ENGINEERS AND DEVICES WORKING TOGETHER Introduction “Idle” opposed to “Running”, in between an event occurred ● We want a cpu to be idle as much as possible without impacting the performance of the system ● The cpuidle framework takes care of entering the idle state depending on the predictable events on the system ● The depth of the sleep will depend on the next event on the system and the latency constraints ● An event makes a CPU to exit its idle state ● Some lecture: − Computing a target residency: https://goo.gl/aywVSI − Sources of wake up: https://goo.gl/zXpQfW
- 3. ENGINEERS AND DEVICES WORKING TOGETHER The idle task ● The idle task is the last task executed on the CPU ● The idle task is an infinite loop − Scheduled out when ‘need_resched’ is set − Otherwise enters the idle function need_resched() idle() schedule()no yes
- 4. ENGINEERS AND DEVICES WORKING TOGETHER Sources of wake up ● Three kinds of interruption: − Inter processor interrupts (IPI) − I/O interrupts − Timer interrupts
- 5. ENGINEERS AND DEVICES WORKING TOGETHER Inter Processor Interrupt ● Used conveniently to inform or remotely execute an action on another processor ● Very few relevant: − Timer broadcasting: used to notify another CPU a timer expired. Pointless if there is an always-on per cpu timer or negligible if the FEAT_DYNIRQ is set for the timer − Rescheduling interrupt: the most important IPI, basically used to wake up an idle CPU when there is a task in the runqueue, mastered by the scheduler. ● Pitfall: it could be a wakeup resulting from an IO completion where the interrupt was received on another CPU
- 6. ENGINEERS AND DEVICES WORKING TOGETHER Timer ● Per cpu timer: − a private irq line ● Global timer: − Can wakeup any CPUs randomly but: ● in practical the DYNIRQ feat will make it to expire on the CPU with the earliest timer ● the cpumask is usually set to CPU0 ● The next event is predictable on this device
- 7. ENGINEERS AND DEVICES WORKING TOGETHER I/O interrupt ● Some are coming from devices with internals communication: − MMC, I2C, SATA, GPU, … − May be they have repeating patterns ? ● And others from external events: − Keyboard, network, mouse, … − May be they can have repeating patterns ? ● Intuitively the former can have a burst of interrupt activity followed by long quiescent point, the latter can be considered as slow devices and a prediction failure has a negligible impact
- 8. ENGINEERS AND DEVICES WORKING TOGETHER Current approach : the glitch ● Measures how long it has been in the idle function ● Statistics are done on the idle durations ● Reminder: a CPU is woken up by: − Timers, IPI and device interrupts ● So the statistics are trying to predict the next interrupt but that includes: ○ The next timer expiration !? ○ The next scheduler decision !?
- 9. ENGINEERS AND DEVICES WORKING TOGETHER Current approach: the soup ● The governors are mixing different kind of information ○ Deterministic and non deterministic ● It is impossible to identify the source of the wake up and: − Discard some events like the timer ones − Identify if the observed interrupt is the expected one − Create a feedback loop prediction vs observation
- 10. ENGINEERS AND DEVICES WORKING TOGETHER Let’s analyze the devices ● Recipe: − Using well known benchmarks − Pin the observed interrupt on a specific CPU − Used trace-cmd with the sequence ‘start’, <cmd>, ‘stop’, ‘extract’ in order to prevent writes to the disk during the test − For the observed interrupt, measure the interval between the occurrences and their frequencies ● Intervals representation gives the chronology of the events and the repeating patterns ● Frequencies give the probabilities − A peak means is a high probability
- 11. ENGINEERS AND DEVICES WORKING TOGETHER MMC / SD card ● Writing on the SD card with sysbench test − sysbench --test=fileio --file-test-mode=seqrewr --file-total-size=128M run
- 12. ENGINEERS AND DEVICES WORKING TOGETHER SATA / SDD ● Writing on the SSD 6Gb/s with sysbench test − sysbench --test=fileio --file-test-mode=seqrewr --file-total-size=128M run
- 13. ENGINEERS AND DEVICES WORKING TOGETHER WiFi ● Copying a file with WiFi from Internet − ketchup v4.10
- 14. ENGINEERS AND DEVICES WORKING TOGETHER Ethernet / Gb ● Copying a file 1,8GB with a 1Gb/s ethernet on a local network via NFS − cp /net/shared/myfile /tmp
- 15. ENGINEERS AND DEVICES WORKING TOGETHER Console / UART ● Writing to the console via UART − dmesg
- 16. ENGINEERS AND DEVICES WORKING TOGETHER SATA / HDD ● Writing on a HDD 3Gb/s with sysbench test − sysbench --test=fileio --file-test-mode=seqrewr --file-total-size=128M run
- 17. ENGINEERS AND DEVICES WORKING TOGETHER Observation ● The frequencies show the peaks − Several peaks => there is a pattern − A single peak => averaging and discarding anomalies is enough ● The intervals are stable, we can see a pattern for some devices − Visible with kernelshark and on the graphics
- 18. ENGINEERS AND DEVICES WORKING TOGETHER Hypothesis ● If we can find the pattern of the interrupts, then we can predict the next event more accurately for each interrupt − What algorithm ? ● Single peak can be computed with a simple average on the last 32 samples (rule of thumb) assuming it is a normal law ● Taking the earliest next events from all interrupts gives the next event
- 19. ENGINEERS AND DEVICES WORKING TOGETHER Challenges ● A pattern rather than a periodic interval could be hard to predict ● The implementation is full of pitfalls − Missing information (IPI IO, per cpu timer flag, etc ...) − Mathematic with large numbers − The idle place is very special (lock, rcu, interrupt, ...) ● Statistics computation consumes energy and takes time − What is the best trade-off ? ● New tools to do an instrumentation (prediction accuracy, spot where the problem is, …)
- 20. ENGINEERS AND DEVICES WORKING TOGETHER Status TBD ● Video ● Audio ● Idle ● User Interface ● Browsing
- 21. ENGINEERS AND DEVICES WORKING TOGETHER What’s next ? ● Change the scheduler to be more energy aggressive ● Prevent to wake up an idle CPU which did not reach its target residency ● IPI are also used to wake up a CPU after an IO completion ○ How to discriminate ? ○ Change the blocked task on IO wake up path