This article was peer reviewed by Dan Prince. Thanks to all of SitePoint’s peer reviewers for making SitePoint content the best it can be!
Over the past decade, browser vendors have introduced various new APIs that enable us as programmers to create richer and more fluent experiences. One of these is the getUserMedia
API, which enables access to the user’s audio and video devices. However, it’s still not quite there yet in terms of browser compatibility.
With this in mind, Adam Wróbel wrote JpegCamera, a library that takes into account the different caveats among browsers for interacting with user’s camera and provides fallbacks for those cases where access to client’s media is not supported.
In this article we’ll see how by using JpegCamera, together with HTML canvas
element capabilities, we can build a clone of Instagram’s Layout app:
The demo Layout-like application
The source code for the demo can be downloaded from Github.
The JpegCamera Library
JpegCamera allows you to access the user’s camera as part of your application, gracefully degrading to a Flash fallback if the browser does not support getUserMedia()
.
The first step to is to include the necessary scripts in your project.
The library depends on the SWF Object and Canvas to Blob libraries, both of which come as part of the zip download from the project’s Github page. However, in the same zip there’s a with dependencies version of the script, which provides the same functionality as having the three scripts loaded in the page.
With this in mind, you can either include the three needed scripts.
<script src="/jpeg_camera/swfobject.min.js" type="text/javascript"></script>
<script src="/jpeg_camera/canvas-to-blob.min.js" type="text/javascript"></script>
<script src="/jpeg_camera/jpeg_camera.min.js" type="text/javascript"></script>
Or just go with the one script alternative.
<script type="text/javascript" src="js/libs/jpeg_camera/jpeg_camera_with_dependencies.min.js"></script>
For production environments, the later seem to be the way to go, unlike during development.
Once the library is included you can use the global JpegCamera object to check the camera availability, and choose how to manage the fallback if not.
If the access is granted, you can setup a listener for when the camera is ready with the JpegCamera()
constructor.
The JpegCamera()
constructor takes a CSS selector as an argument which should identify the container to be used for the camera stream.
The snippet below shows the code that does this:
(function() {
if(!window.JpegCamera) {
alert('Camera access is not available in your browser');
} else {
JpegCamera('.camera')
.ready(function(resolution) {
// ...
}).error(function() {
alert('Camera access was denied');
});
}
})();
This way, you can setup your application to only start once the camera is ready, or let the user know that they either require a different browser or need to enable camera access for the application.
Inside the ready
callback function, the device’s camera resolution is passed as the first argument. This can be useful if the application we are building relies on the device’s camera quality (i.e.: to make HD capture available or not).
Meanwhile the error
callback receives as an argument a string
message explaining what happened. If you need to show the user an explanation in case of an error, you can use the message the library supplies.
In addition to this, the JpegCamera API provides the following methods:
capture()
: This is the method that takes a picture. It returns the image itself as aSnapshot
object (the class thatJpegCamera
uses for images).show()
: Once you take the picture, theSnapshot
object that you obtain allows you to display the image in the page, by invoking itsshow()
method. The image will be displayed inside the same container you specified when initializing the camera.showStream()
: If a snapshot is currently displayed in the container,showStream()
hides the image and displays the stream.getCanvas()
: Takes a callback function as a parameter, which will receive as an argument thecanvas
element with the captured image.
Let’s dive into an example application that illustrates what JpegCamera allows us to do.
Building the Application
The demo application emulates (sort of) what Layout does: it allows the user to take photos and generates new images by combining them. In our version, the combined images can be downloaded by clicking on them.
The application structure is based on the Module Pattern. This pattern gives us a couple of benefits:
- It allows to have a clear separation between each of the application components.
- It keeps our global scope clean by only exposing methods and properties that are strictly required by the others. In other words, we get to use private attributes.
You’ll notice that I pass three parameters into the self invoked functions:
(window, document, jQuery)
And these arguments are received:
function(window, document, $)
The reason for passing window
and document
is for minification purposes. If we pass these as arguments, then each of them can be replaced for a single character. If we had just referenced these global objects directly, the minifier would not be able to substitute them with shorter names.
With jQuery
, we do it to avoid conflicts with other libraries that might also using $
as their main function (i.e.: Prototype).
At the top of the Layouts
and Custom
modules you’ll see something along these lines:
if(!window.LayoutApp) {
window.LayoutApp = {};
}
This is for two reasons:
- We prevent the modules from generating errors in case we did not include the scripts properly in
index.html
. - We keep our global scope clean by making the modules part of a main one and only available for it once the application starts.
The application logic is divided into three modules:
- The App module
- The Layouts module
- The Custom module
These three modules together with our libraries must be included in our index.html
as follows:
<!-- index.html -->
<script type="text/javascript" src="js/libs/jquery-1.12.1.min.js"></script>
<script type="text/javascript" src="js/libs/jpeg_camera/jpeg_camera_with_dependencies.min.js"></script>
<script type="text/javascript" src="js/src/custom.js"></script>
<script type="text/javascript" src="js/src/layouts.js"></script>
<script type="text/javascript" src="js/src/app.js"></script>
And there’s one more small piece of code to start the application.
<!-- index.html -->
<script type="text/javascript">
(function() {
LayoutApp.init();
})();
</script>
Now, let’s review the modules one by one.
The App module
This module holds the main application logic. It manages the user interaction with the camera, generates the layouts based on the pictures taken and enables the user to download the generated images.
Everything starts in the App module, with the init
method.
// App module (app.js)
initCamera = function () {
if (!window.JpegCamera) {
alert('Camera access is not available in your browser');
} else {
camera = new JpegCamera('#camera')
.ready(function (resolution) {})
.error(function () {
alert('Camera access was denied');
});
}
},
bindEvents = function () {
$('#camera-wrapper').on('click', '#shoot', capture);
$('#layout-options').on('click', 'canvas', download);
};
init: function () {
initCamera();
bindEvents();
}
When invoked, ìnit()
kicks off the application by calling the following methods:
initCamera()
starts the camera, if available, or shows an alert.bindEvents()
sets up the necessary event listeners: 1. The first one to take the photos upon clicking the Shoot button. 2. The second one to generate the download when clicking on one of the combined images.
capture = function () {
var snapshot = camera.capture();
images.push(snapshot);
snapshot.get_canvas(updateView);
},
When the user clicks on Shoot, capture()
is invoked. capture()
uses Snapshot’s class method getCanvas()
passing as the callback updateView()
function.
updateView = function (canvas) {
canvas.selected = true;
canvases.push(canvas);
if (!measuresSet) {
setCanvasMeasures(canvas);
measuresSet = true;
}
updateGallery(canvas);
updateLayouts(canvas);
},
In turn, updateView()
caches the new canvas
object (see updateGallery()
) and updates the layouts with the new image by invoking updateLayouts()
, which is the method that does the magic.
updateLayouts()
relies on the following three methods:
setImageMeasures()
: This one defines an adequate width and height for the images, considering how many have been taken.setSourceCoordinates()
: By checking the image measurements, this returns the coordinates for the center of the image.setTargetCoordinates()
: This one takes into account the index of the image to be drawn and returns the coordinates of where the images will be drawn on the target canvas.
In addition to this, calculateCoeficient()
takes care of keeping the proportions between the orginal image and the one to be generated, by comparing the source and the target canvas measures.
Finally, updateLayout()
draws the image in the new canvas by using context.drawImage()
with the data from the four functions above. The implementation to be used will be the one that uses its eight parameters. Meaning that we specify the source coordinates, source measures, target coordinates and target measures.
The Layouts module
The Layouts
module provides the basic layout data, together with some helper functions.
Since we want to keep our scopes clean and just share with the other modules what’s strictly necessary, the Layouts
module gives access to the attributes the App
module needs through its getters.
// Layouts module (layouts.js)
var CANVAS_MAX_MEASURE = 200,
LAYOUT_TYPES = {
HORIZONTAL: 'horizontal',
VERTICAL: 'vertical'
},
LAYOUTS = [
{
type: LAYOUT_TYPES.VERTICAL
},
{
type: LAYOUT_TYPES.HORIZONTAL
}
];
return {
getCanvasMaxWidth: function() {
return CANVAS_MAX_MEASURE;
},
getLayouts: function() {
return LAYOUTS.concat(Custom.getCustomLayouts());
},
isHorizontal: function(layout) {
return layout.type === LAYOUT_TYPES.HORIZONTAL;
},
isVertical: function(layout) {
return layout.type === LAYOUT_TYPES.VERTICAL;
},
isAvailable: function(layout, totalImages) {
return !layout.minImages || layout.minImages <= totalImages;
}
}
As seen above, none of the modules can mutate what is inside the Layouts
module, but all that’s needed for the application to work is readily available.
Here’s what each of these methods contribute to the application:
getCanvasMaxWidth()
: In order to keep the images tidy I determined a default width for them and assigned it toCANVAS_MAX_MEASURE
. This value is used in theApp
module to define the combined image measures. See the snippet below for the actual math within theApp
module.
// App module (app.js)
setCanvasMeasures = function (canvas) {
measures.height = canvas.height * MAX_MEASURE / canvas.width;
},
This way our combined images can have any measure we like, independently from how big the ones we get from JpegCamera are and we prevent any stretching or shrinking by maintaining the aspect ratio from the picture just taken.
getLayouts()
: Returns the layouts that generate the combined images from the pictures taken by the user. It returns both the application default layouts, together with the custom ones that can be added to theCustom
module (more on this later).isHorizontal()
andisVertical()
: The default layout in the application are defined by setting itstype
attribute, which takes its values from theLAYOUT_TYPES
. By receiving alayout
object as an argument and relying on this constant, these two methods evaluatelayout.type === LAYOUT_TYPES.HORIZONTAL
andlayout.type === LAYOUT_TYPES.VERTICAL
. Based on the return values of these functions, theApp
module defines the measures, source coordinates and target coordinates for the combined images.isAvailable()
: Depending on how many images the user took and considering the layout’sminImages
attribute, this function determines if the layout should be rendered or not. If the user has taken as many images or more than the ones set as the minimum, then the layout will be rendered. Otherwise, if the user has not taken as many photos or the layout does not have aminImages
attribute defined, then thecombined
image will be generated.
The Custom module
The Custom
module allows the addition of new layouts with their own implementation of the applications main three methods: setImageMeasures()
, setSourceCoordinates()
, and setTargetCoordinates()
.
This can be achieved by having a new layout object added to the Custom
module’s CUSTOM_LAYOUTS
array with its own implementation of the above three methods.
// Custom module (custom.js)
var CUSTOM_LAYOUTS = [
/**
* Place your custom layouts as below
*/
// ,
// {
// setImageMeasures: function (layout, targetCanvas, imageIndex) {
// return {
// height: 0,
// width: 0
// }
// },
// setSourceCoordinates: function (canvas, layout, imageWidth, imageHeight, imageIndex) {
// return {
// x: 0,
// y: 0
// }
// },
// setTargetCoordinates: function (targetCanvas, layout, imageWidth, imageHeight, imageIndex) {
// return {
// x: 0,
// y: 0
// }
// }
// }
];
Each of the overriden functions in the application, will check if the layout being drawn contain a function for this.
See how it’s done in App.setImageMeasures()
:
// App module (app.js)
setImageMeasures = function (layout, targetCanvas, imageIndex) {
if (isFunction(layout.setImageMeasures)) {
return layout.setImageMeasures(layout, targetCanvas, imageIndex);
} else {
if(Layouts.isVertical(layout)) {
return {
width: $(targetCanvas).width(),
height: $(targetCanvas).height() / images.length
};
} else if(Layouts.isHorizontal(layout)) {
return {
width: $(targetCanvas).width() / images.length,
height: $(targetCanvas).height()
};
}
return {
width: $(targetCanvas).width(),
height: $(targetCanvas).height()
};
}
}
Here we simply check if the custom layout has its own implementation of a function to define the image measures and, if so, invoke it.
This is achieved by the isFunction()
helper, which checks if the argument received is actually a function by checking its type.
// App module (app.js)
isFunction = function(f) {
return typeof f === 'function';
}
If the current module does not contain its own implementation of setImageMeasures()
, the application goes on and sets the measures according to the layout type (either HORIZONTAL
or VERTICAL
).
The same flow is followed by setSourceCoordinates()
and setTargetCoordinates()
.
The new layout can determine the size of the section to be cropped from the image taken, from which coordinates, and where it will be placed on the target canvas.
One important detail is to keep in mind that the custom layout methods must return objects with the same attributes as the original methods do.
To be clearer, your custom implementation of setImageMeasures()
should return something in this format:
{
height: 0, // height in pixels
width: 0 // width in pixels
}
Creating a custom layout
Let’s have a go at creating a custom layout. You can see the full code listing for this file here.
As seen in the Layouts
module section, layouts can have a minImages
attribute defined. In this case, let’s set it to 3. Let’s also have the first image taken cover 60% of the target canvas, while the next two will split the remaining 40%:
{
minImages: 3,
imageData: [
{
widthPercent: 60,
heightPercent: 100,
targetX: 0,
targetY: 0
},
{
widthPercent: 20,
heightPercent: 100,
targetX: 120,
targetY: 0
},
{
widthPercent: 20,
heightPercent: 100,
targetX: 160,
targetY: 0
},
],
// ...
To achieve this, let’s apply a simple rule of three, using the targetCanvas
measures:
// Custom module (custom.js)
setImageMeasures: function (layout, targetCanvas, imageIndex) {
var imageData = this.imageData[imageIndex];
if( imageData) {
return {
width: imageData.widthPercent * $(targetCanvas).width() / 100,
height: imageData.heightPercent * $(targetCanvas).height() / 100
};
}
return {
height: 0,
width: 0
}
},
Since all of the functions receive as an argument the number of the image (imageIndex
) currently being processed, we can arbitrarily determine the size, the source coordinates for cropping, and the coordinates where original image’s section will be placed in the target canvas for each of the photos.
In the case where there’s no data associated with a particular imageIndex
, we can just return an object with both of the attributes set to 0. This way, if the user takes more pictures than those that are defined in the custom layout, the combined image will still look good.
Let’s override the other two functions:
setSourceCoordinates() Given that we want to include the center of the image with all of its vertical content, we will return an object with x set to 50 and y set to 0.
setSourceCoordinates: function (canvas, layout, imageWidth, imageHeight, imageIndex) {
return {
x: 50,
y: 0
}
},
setTargetCoordinates() Since we know the canvas’ measures, let’s just manually define where they’d be placed in the target canvas.
setTargetCoordinates: function (targetCanvas, layout, imageWidth, imageHeight, imageIndex) {
var imageData = this.imageData[imageIndex];
if (imageData) {
return {
x: imageData.targetX,
y: imageData.targetY
}
}
return {
x: 0,
y: 0
}
}
As you may agree, there’s plenty of room for improvement on this example, but this should be enough to get you started.
Conclusion
As we saw, JpegCamera takes the pain out of using the user’s camera in your application without needing to worry about cross-browser compatibility.
Including it as part of your project is as simple as adding the needed scripts to the page, and to use it requires understanding just four API methods. Writing a fun application takes only slightly more than a couple hundreds lines of code!
How about you, have you ever needed to work with user’s media? Are you keen to try implementing your own layout? Let me know in the comments!
Frequently Asked Questions about JPEGCamera and Canvas
How can I use JPEGCamera with HTML5 Canvas?
JPEGCamera is a JavaScript library that allows you to access the user’s camera and capture photos. To use it with HTML5 Canvas, you first need to include the JPEGCamera library in your HTML file. Then, you can use the ‘get_canvas’ method to get the captured image as a canvas. You can then manipulate this canvas using the HTML5 Canvas API. For example, you can draw shapes on it, apply filters, or even combine it with other canvases.
What are the benefits of using JPEGCamera with Canvas?
Using JPEGCamera with Canvas allows you to manipulate the captured images in real-time. You can apply filters, draw shapes, or combine images. This can be useful for a variety of applications, such as photo editing apps, augmented reality apps, or even games.
How can I display the captured image on the canvas?
Once you have captured an image using JPEGCamera, you can use the ‘get_canvas’ method to get the image as a canvas. You can then use the ‘drawImage’ method of the Canvas API to draw this canvas onto another canvas. This allows you to display the captured image on the canvas.
Can I use JPEGCamera with other JavaScript libraries?
Yes, JPEGCamera can be used with other JavaScript libraries. It is a standalone library, so it does not have any dependencies. However, it can be easily integrated with other libraries if needed. For example, you can use it with jQuery to easily manipulate the DOM, or with Three.js to create 3D graphics.
How can I handle errors when using JPEGCamera?
JPEGCamera provides a ‘catch’ method that you can use to handle errors. This method is called when an error occurs during the execution of the library. You can use this method to display an error message to the user, or to perform some other action.
Can I use JPEGCamera on mobile devices?
Yes, JPEGCamera is compatible with most modern mobile devices. However, the user’s device must have a camera and the browser must support the getUserMedia API. If these requirements are not met, JPEGCamera will not work.
How can I save the captured image?
After capturing an image with JPEGCamera, you can use the ‘snapshot’ method to get the image as a JPEG. You can then send this JPEG to a server to be saved. Alternatively, you can use the ‘toDataURL’ method of the Canvas API to get the image as a data URL, which can be saved directly to the user’s device.
Can I use JPEGCamera with WebGL?
Yes, you can use JPEGCamera with WebGL. After capturing an image, you can get it as a canvas using the ‘get_canvas’ method. You can then use this canvas as a texture in WebGL.
How can I apply filters to the captured image?
After capturing an image with JPEGCamera, you can get it as a canvas using the ‘get_canvas’ method. You can then use the Canvas API to manipulate the pixels of this canvas and apply filters. For example, you can change the color balance, apply a sepia effect, or blur the image.
Can I use JPEGCamera in a commercial project?
Yes, JPEGCamera is licensed under the MIT license, which allows you to use it in both commercial and non-commercial projects. However, you must include the original copyright notice and disclaimer in any copies of the software.
I am a Technical Lead working for Alphero, who's still getting used to constantly windy Wellington. I like twitting @loqtor when feeling like saying something short, or writing at Cause I can write when 140 characters would not make it.