Cours real-time multimedia composition using Lua, tutoriel & guide de travaux pratiques en pdf.
ABSTRACT
In this paper, a new interface for programming multimedia compositions in Max/MSP/Jitter using the Lua scripting language is presented. Lua is extensible and efficient making it an ideal choice for designing a programmatic interface for multimedia compositions. First, we discuss the distinctions of graphical and textual interfaces for composition and the requirements for a productive compositional workflow, and then we describe domain specific implementations of Lua bindings as Max externals for graphics and audio in that order.
Categories and Subject Descriptors
H.5.1 [Information Interfaces and Presentation] Multimedia Information Systems – animations; H.5.5 [Information Interfaces and Presentation] Sound and Music Computing methodologies & techniques, modeling, signal synthesis and processing; I.3.7 [Computer Graphics] Three-Dimensional Graphics and Realism – animation; J.5 [Arts and Humanities] –arts, fine arts and performing, music.
Keywords: Lua, Max/MSP, Jitter, OpenGL, audiovisual composition.
INTRODUCTION
The contemporary digital artist can choose amongst many software tools to express his or her idea, however frequently there is a great challenge in manipulating such tools to express complex time-variant structures. Real-time multimedia software must handle continuous flows of interacting heterogeneous data streams yet provide a flexible interface for describing interactions between media, all the way from lowlevel data and memory processing up to the more abstract structures of content and form.
Max/MSP/Jitter
The Max/MSP/Jitter environment is a popular choice for constructing complex interactive digital media works.
Amongst its strengths are bindings to many different media types and protocols and the open-ended philosophy: Max carries few preconceptions regarding how or for what it should be used.
Lua
Lua is a powerful, efficient scripting language based on associative tables and functional programming techniques.
Lua is ideally suited for real-time multimedia processing because of its flexible semantics and fast performance; it is frequently used for game logic programming (e.g. World of Warcraft [24]) and application extension (e.g. Adobe Lightroom).
jit.gl.lua and lua~
The jit.gl.lua (for OpenGL graphics) and Lua~ (for audio DSP) extensions allow a new range of expressive control in whicharbitrary, dynamic data structures and behaviors can be generated, whilst interfacing with the convenient graphical interface and libraries of Max/MSP/Jitter. The development of these extensions has been driven by the presenters’ artistic goals and requirements.
MULTIMEDIA COMPOSITION ENVIRONMENTS
Digital media composition
Designing an environment for multimedia composition in the digital domain requires constructing a model of multimedia representation. The ideal specificity of a model depends upon the problem it is trying to solve, however digital media composition is a highly unconstrained problem. A good model should balance design for efficiency with minimization of preconceptions about its potential usage.
Building a multimedia composition where audio, video, 3D graphics, MIDI, OSC, and other hardware and communication protocols need to be pulled together in real-time to realize the composition requires a good amount of technical knowledge as well as a large, modular code base. To handle the complexity of this task and allow rapid sketching of compositional ideas, a number of general purpose multimedia composition environments have been developed of which the Max/PD family [27, 19] is one of the most widely used. With a basic knowledge of the environment, a user can link together sonic and visual processes and interactively construct and audiovisual composition.
Using visual data-flow programming for
multimedia composition the Data Flow Architecture pattern [14, 20] performs sorted operations on data elements (tokens) through a directed graph of modules (nodes) connected via ports (arcs). Modules read tokens on their input ports, process them according to a module-specific internal algorithm, and then write tokens into their output ports. Tokens can flow through the network in regular rates (stream flow) or irregular rates (event flow).
Lua used as a Domain Specific Language for Multimedia
For real-time multimedia composition, interpreted languages must be expressive, efficient to work with, have a low computational cost, and be flexible in terms of how they fit into the compositional workflow. The Lua scripting language has become widely adopted by the game developer community for precisely these reasons. Lua’s authors describe Lua as an extension language [9] specifically designed to be embedded within host programs and extended by domain-specific APIs for interaction with the host program. Lua meets the needs of an extension language by providing good data description facilities (associative tables that can also function as arrays), using clear and simple syntax (suitable for non-professional users), offering flexible semantics for extensibility, being small and easy to embed, and without needing to satisfy generic programming constraints such as static type-checking.
………
Si le lien ne fonctionne pas correctement, veuillez nous contacter (mentionner le lien dans votre message)Real-time multimedia composition using lua (319 KO) (Cours PDF)
