HTML5 video filters
0 likes1,195 views
The document discusses various methods for applying video filters in HTML5, outlining options such as SVG, CSS, WebGL, and Canvas, noting their browser support and developer familiarity. It emphasizes the use of the Canvas element due to its good support and provides programming tips for optimizing performance. Additionally, it highlights advanced topics like typed arrays, buffers, and web workers for enhanced power and efficiency in video processing.
![PROGRAMMING TIPS
consider not using frameworks or polyfills
access DOM as little as possible
reduce the number of dots
cache everything (i said EVERITHING!)
don't write 2 times the same thing
use while(i--) insead of for(i++)
multiply instead of divide
requestAnimationFrame()
['a','b'].join('') instead of 'a'+'b'
bitwise operations when convenient (i.e. floor if n>=0)
O'Reilly - High Performance JavaScript
jsPerf, StackOverflow, Google](https://image.slidesharecdn.com/jurehtml5videofilters-141111041646-conversion-gate01/85/HTML5-video-filters-11-320.jpg)
![TYPED ARRAYS
Direct access to RAM (IE10+)
// Floating point arrays.
var f64 = new Float64Array(8);
var f32 = new Float32Array(16);
// Signed integer arrays.
var i32 = new Int32Array(16);
var i16 = new Int16Array(32);
var i8 = new Int8Array(64);
// Unsigned integer arrays.
var u32 = new Uint32Array(16);
var u16 = new Uint16Array(32);
var u8 = new Uint8Array(64);
var pixels = new Uint8ClampedArray(64);
Example
var a = new Uint8Array(64); // length = 64 // 0 < value < 2^8 ;
a[0] = 300 ;
console.log (a[0]) // 44 = 300 - 256](https://image.slidesharecdn.com/jurehtml5videofilters-141111041646-conversion-gate01/85/HTML5-video-filters-14-320.jpg)
![BUFFERS AND VIEWS
var buffer = new ArrayBuffer(16); // 16 byte = 128 bit of RAM
// to access it, you need to create a view
var int32View = new Uint32Array(buffer); // length = 4 (128/32)
// each element = 0 < value < 2^32
// you can create multiple views of the same buffer
var int8View = new Uint8Array(buffer); // length = 16 (128/8)
// each element = 0 < value < 2^8
// what does it mean?
int32View[0]=300;
console.log(int32View[0]); // 300
console.log(int32View[1]); // 0
console.log(int8View[0]); // 44
console.log(int8View[1]); // 1
// it means that the Endianess is "Little-endian" (first byte = less significant)](https://image.slidesharecdn.com/jurehtml5videofilters-141111041646-conversion-gate01/85/HTML5-video-filters-15-320.jpg)
![YOU ARE ALREADY USING THEM
Canvas 2D
alert(canvasElement.getContext('2d').getImageData(0,0,w,h).data);
// in most recent browsers, "data" is an [object Uint8ClampedArray]
// in older browsers, "data" is an [object CanvasPixelArray]
// the cause is an HTML5 spec change
var a = new Uint8ClampedArray(2); // length = 2 < value < 2^8 ;
a[0] = 300 ;
console.log (a[0]) // 255
console.log (a[1]) // 0
XMLHttpRequest2
File APIs
Media Source API
WebGL
Transferable objects
Binary WebSockets](https://image.slidesharecdn.com/jurehtml5videofilters-141111041646-conversion-gate01/85/HTML5-video-filters-16-320.jpg)
![WHAT IS THE (COLOR)MATRIX?
var brightness_colorMatrix = [
// r g b a
/*R*/ 1, 0, 0, x,
/*G*/ 0, 1, 0, x,
/*B*/ 0, 0, 1, x,
/*A*/ 0, 0, 0, 1
];
// R = r*1 + g*0 + b*0 + a*x ;
// G = r*0 + g*1 + b*0 + a*x ;
// B = r*0 + g*0 + b*1 + a*x ;
// A = r*1 + g*0 + b*0 + a*1 ;](https://image.slidesharecdn.com/jurehtml5videofilters-141111041646-conversion-gate01/85/HTML5-video-filters-26-320.jpg)
HTML5 video filters
- 1. HTML5 VIDEO FILTERS IN SEARCH OF THE DRAGON BALLS Created by [email protected]
- 2. OBJECTIVE Show in a browser a video with the following controls: Grayscale Brightness Contrast RGB channels
- 3. LOOKING FOR AN IDEA 1. The mysterious SVG filters? ✗ potentially the best option, but 2014 is too early because... ✗ not so good browsers' support ✗ IE9- (XP/Vista) don't support SVG at all 2. The lightweight CSS filters? ✗ bad browsers' support ✗ no support for feColorMatrix SVG filter 3. The powerful WebGL? ✗ bad browsers' support ✗ unfamiliar enviroment for most developers ✗ not W3C, depends from graphic board's drivers 4. The hungry Canvas? ✓ good browsers' support (IE9+) ✓ polyfills for IE8 (but don't rely on them) ✓ familiar enviroment for most developers
- 4. A VIDEO IN A CANVAS var videoElement = document.getElementById('myVideo') ; var canvasElement = document.getElementById('myCanvas') ; var canvasContext = canvasElement.getContext('2d') ; canvasContext.drawImage(videoElement,0,0,width,height) ; // loop
- 5. DEMO VIDEO 854 × 480 = 409.920 px × 4 channels = 1.639.680 byte CANVAS Red 80% Green 80% Blue 80% Contrast 80% Brightness 80% Black & white FPS 13.9
- 7. LET'S TRY FULL HD
- 8. DEMO FULL HD
- 9. WHAT DO WE NEED?
- 10. MORE POWER!
- 11. PROGRAMMING TIPS consider not using frameworks or polyfills access DOM as little as possible reduce the number of dots cache everything (i said EVERITHING!) don't write 2 times the same thing use while(i--) insead of for(i++) multiply instead of divide requestAnimationFrame() ['a','b'].join('') instead of 'a'+'b' bitwise operations when convenient (i.e. floor if n>=0) O'Reilly - High Performance JavaScript jsPerf, StackOverflow, Google
- 12. AND NOW, WHAT DO WE NEED?
- 13. MORE POWER!
- 14. TYPED ARRAYS Direct access to RAM (IE10+) // Floating point arrays. var f64 = new Float64Array(8); var f32 = new Float32Array(16); // Signed integer arrays. var i32 = new Int32Array(16); var i16 = new Int16Array(32); var i8 = new Int8Array(64); // Unsigned integer arrays. var u32 = new Uint32Array(16); var u16 = new Uint16Array(32); var u8 = new Uint8Array(64); var pixels = new Uint8ClampedArray(64); Example var a = new Uint8Array(64); // length = 64 // 0 < value < 2^8 ; a[0] = 300 ; console.log (a[0]) // 44 = 300 - 256
- 15. BUFFERS AND VIEWS var buffer = new ArrayBuffer(16); // 16 byte = 128 bit of RAM // to access it, you need to create a view var int32View = new Uint32Array(buffer); // length = 4 (128/32) // each element = 0 < value < 2^32 // you can create multiple views of the same buffer var int8View = new Uint8Array(buffer); // length = 16 (128/8) // each element = 0 < value < 2^8 // what does it mean? int32View[0]=300; console.log(int32View[0]); // 300 console.log(int32View[1]); // 0 console.log(int8View[0]); // 44 console.log(int8View[1]); // 1 // it means that the Endianess is "Little-endian" (first byte = less significant)
- 16. YOU ARE ALREADY USING THEM Canvas 2D alert(canvasElement.getContext('2d').getImageData(0,0,w,h).data); // in most recent browsers, "data" is an [object Uint8ClampedArray] // in older browsers, "data" is an [object CanvasPixelArray] // the cause is an HTML5 spec change var a = new Uint8ClampedArray(2); // length = 2 < value < 2^8 ; a[0] = 300 ; console.log (a[0]) // 255 console.log (a[1]) // 0 XMLHttpRequest2 File APIs Media Source API WebGL Transferable objects Binary WebSockets
- 17. AND NOW, WHAT DO WE NEED?
- 18. MORE POWER!
- 19. WEB WORKERS (IE10+) Execute JavaScript in background (multithreading!!!) Honestly, i had no time to try them :P demo by Conor Buckley using getUserMedia() http://bit.ly/10YhHoK
- 20. LET'S TRY ALL THIS STUFF (except for web workers)
- 21. DEMO FULL HD OPTIMIZED
- 22. AND NOW, WHAT DO WE NEED?
- 23. MORE POWER!
- 24. PIXI.JS 2D WebGL framework with canvas fallback O.O pixijs.com (video with audio will autoplay on next slide)
- 25. DEMO FULL HD WEBGL
- 26. WHAT IS THE (COLOR)MATRIX? var brightness_colorMatrix = [ // r g b a /*R*/ 1, 0, 0, x, /*G*/ 0, 1, 0, x, /*B*/ 0, 0, 1, x, /*A*/ 0, 0, 0, 1 ]; // R = r*1 + g*0 + b*0 + a*x ; // G = r*0 + g*1 + b*0 + a*x ; // B = r*0 + g*0 + b*1 + a*x ; // A = r*1 + g*0 + b*0 + a*1 ;
- 27. THANK YOU BY GIORGIO . BEGGIORA @ ARTIGIANI DEL WEB . IT