Talk Node.js with the Experts – How it all Went Down

So… I guess now that the guys are all here I’ll make a formal introduction. Giovanni, Don and Mark are the team behind our very soon to be released Node.js Jump Start book (which is the first in a new series that we’re introducing). Thanks for your time this morning guys.

I’m Community Manager at SP and I’ll be moderating the session.

whats everyone experience with node?

If you haven’t taken part in one of these sessions before, it’s a bit of a free for all. Jump in with a question at any time & I’ll make sure that I stay on top of any that might get lost in the noise.

What IS node?

@suniled none as yet but seen on some job postings so wanting to learn more

http://nodejs.org/

What apps have you guys built that are in production?

freshman here (haven’t even made a hello world yet)

I’m working on a rather large(ish) project right now…and I have a question for any expert who knows about transferring Backbone.js application state using Socket.IO.

suniled, It has its idiosyncrasies but once you get used to them it’s a very pleasant programming environment

experimented only

JerryE, it’s a platform for building fast and scalable network applications. It’s generally used for real time web apps

Node.js is a wrapper around Google’s V8 JavaScript engine

Been working on node for about 6months, love it !! building simple sites to chat boxes… nothing in production… all learning

Want to get into the real-time applications…

It allows you to interact with the system the way any normal application does

So Node.js doesn’t have to be limited to just web apps

@markdalgleish That’s a good point. I’ve made a few small tools that run cross platform

@mb4, I was in charge of technology for HouseFix. Node.js did everything we wanted technologically but unfortunately the startup has decided to wind down for various business reasons

Ive heard a few critics of node.js, and watched a few videos on YouTube, Im sure you know the ones I mean?

I see LinkedIn are now using it. Is it a case that it’s really built for higher traffic sites rather than normal static type sites?

@rettal, to which specific criticisms are you referring?

I wondering if some of the experts can talk about the future of node… like what can we expect when 1.0 comes out?

@dmon well, you can handle more easily thousands of requests because of its async nature

@dmon, it can be used for any site but its real competitive advantages shine when it comes to real time web applications

@dmon Node.js can be used as for static sites, but that’s not really playing to its strengths

spefically with regard to the event loop and how we as programmers are left to handle more lower level stuff whereas these problems have alreadybeen solved?

Where Node is really powerful is in its real-time, event-driven nature

@TJKoury I imagine your question will require quite a bit of to and fro, perhaps it is one that is best for forums?

OK thanks :)

@DonNguyen Thanks mentioning that, I’ll take you up on it :>

Here’s a more general question: what are some strategies to reuse server-side code on the client?

FYI any Node questions can be asked at our JS forum here https://www.sitepoint.com/forums/forumdisplay.php?15-JavaScript-amp-jQuery

@rettal, again do you have any specific examples of the ‘lower level stuff’? I know the callback style has received criticism but there are libraries available that make this read more like the traditional programming style

Thanks

I wouldn’t say that Node.js is very low level. I can see that people can struggle with the callback style

TJKoury, there is a package called dnode https://github.com/substack/dnode It allows for RPC functionality and can be used to share libraries on the client and the server

the more you work on node though the callback style is not that big of a deal IMO

I guess it makes a different way of thinking about working with the web? Rather tahn thinking in terms of “request in, handle, and give an answer”

The callback style forces you to think async from the beginning, it’s where a lot of Node’s power comes from

coding servers / http requests is lower level than most platforms.

@donNguyen I’ll definitely check that out, thanks

for a n00b, is it akin to a stripped down version of php+apache in one but more efficient?

I guess I mean with regard to the non-blocking nature of node.js

@rettal I’m not sure we understand what you’re asking

mb4, that’s true. In an environment such as PHP, you have a separate web server such as Apache. In Node.js, you don’t have this decoupling. Whilst it does force you to create your own server (in under 10 lines) it also gives you full control over the http request response cycle

@dmon In Node, you write the server

There are libraries built around making a standard server easier, but you can make a server that’s much smarter

well, apache will handle that stuff for you, with node,js yours on your own thus increasing complexity but I guess callbacks help with that?

I’ve found that Node is more efficiently integrated with the ‘low-level’ stuff; try watching a file for changes using PHP.

(you can do it, but it will kill your CPU)

The callbacks with system hooks are a huge plus, as is having low-level control over the http server.

dmon, it’s not just a stripped down version. Other big differences would be: single language between front end and back end, NoSQL as opposed to SQL, asynchronous as opposed to synchronous.

OK thanks I thnk I get it now. I had it in my head that it was ‘just’ another Apache server like Nginx in a sense, but clearly not :D

@rettal it’s easier than you think

As someone who is interested in this but has no experience in it. Where do you recommend I start? articles, sites & books?

@donNguyen OK thanks. Are ‘normal’ DBs like MySQL/Postgres available to use too out of the box or is it a case of adding in a library or ?

I think people see the famous seven ( or whatever ) line server and think WOW!, and rightly so but I think there is a lot more to it

here’s a good intro… http://book.mixu.net/

rettal, callbacks are not used for complexity management but rather asynchronous programming. Writing your own server in Node.js is not very complicated at all

… and http://toolbox.no.de/

@suniled thanks

And here are some blogs/sites http://www.nodecloud.org/

I read this early on when I was learning Node.js: http://www.nodebeginner.org/

ok, thanks Don

dmon, MySQL can be installed with a single command from the package system

Keep those close… bounce around… find a project… and nodeOUT

Personally, I learned a lot from trying to solve real problems using Node, throwing myself in the deep end

@rettal I think you’ll find that most of that ‘complexity’ that you’re missing from Apache or whatnot is unnecessary to whatever you’re working on, so you don’t have to manage a huge httpd.conf or whatever (most if which is copied from Apache forums…)

will do!! 4 new bookmarks to read

you just configure what you need!

Also, I’d recommend visiting a local JS meetup and finding a Node.js expert to guide you through the basics

if you don’t have a good understanding on web protocols (like what http means, etc) its quite a steep climb

I found that to be possibly the best way to get up to speed

OK. So I’ve got an API I built for a couple of sites talking to a SAP system, using Node for something like that instead of the current apache/php/curl setup, would that be a good use? Not high traffic, but to start me out in a real-world enviro?

keller, aside from the obligatory plug for Jump Start Node.js, projecteuler.net is my first resource when learning any new programming language

@markdalgleish true… but its not that easy finding one :)

@dmon node is perfect for apis, especially if you talk json

dmon, is it a REST API?

thank you @DonNguyen

@donNguyen Errr… it feeds in/out text files, basically generates order files to send to SAP, gets back a reply file with successfully placed orders [in SAP], out of stock files, etc. either as .txt or .csv

It seems that most people using node for web apps use the Express framework. I haven’t found great example web apps that aren’t buggy as hell. Is it fair to say Node works well as a supporting technology for specific areas it’s well suited to but still doesn’t have a mature web app framework for building the whole thing in Node?

@mb4 I don’t think that build a complete web framework like rail is the objective of node

@mb4 Trello is a great example of a large scale, high traffic site built with Node.js

dmon, Node.js will serve your purpose perfectly well but that’s not the type of project where it really shines over and above PHP. As a learning project it works great

@mb4 node is build for fast and non-blocking application (can be web or not)

oh, I didn’t know Trello is completely in Node. gvvn, that’s my impression too.

It’s certainly possible, and Fog Creek have said how much they enjoy using Node.js: http://blog.fogcreek.com/the-trello-tech-stack/

sstephenson just tweeted this he built https://s3.amazonaws.com/f.cl.ly/items/1z1R100K1O3R2I1I3F43/kinect%20png%20stream.mov https://github.com/sstephenson/kinect

@mb4 I think it really matters what you’re trying to do. If you’re rendering pages on the server for SEO or whatever, you probably want to use something else. For me, since I use Backbone.js for everything and render views on the client, it works great as a data aggregator/cache/validator.

@mb4 geeklist is a node app too. Node gie you the building blocks and the you can build your app as you like

Pulling in data from the Kinect and displaying visualisations. It seems like people are using Node for new things which is kinda cool.

Server Frameworks: http://expressjs.com/ , http://towerjs.org/ , http://railwayjs.com/ (if your

mb4, Express.js is an extremely well tested framework. Designed more to be like Sinatra rather than Rails. I imagine the bugs are more due to the programmer rather than the environment :-) In Jump Start Node.js, we spend quite a bit of time going through test driven design.

@mb4 that’s is great if you want to build a fast and scaling application, instead of a monolithic app with rails (or other frameworks)

Server Frameworks: http://expressjs.com/ , http://towerjs.org/ , http://railwayjs.com/ (if your from ruby), http://flatironjs.org/

if you like streams try https://github.com/mikeal/tako

@donNguyen ok thanks understood

@mb4 My experience with Express has been great. The only real issue I’ve had is with a module that had a memory leak, not with the web framework.

Client Side: http://angularjs.org/ , http://emberjs.com/ , http://backbonejs.org/

@gvnn, I’d never built a monolithic anything, just multiple small apps.

Thanks for the thoughts on frameworks, keen to check out your book when it’s released.

@mb4 well then node can be a good option… I certainly suggest node for APIs and process of stream of data

Node is great for making lightweight but powerful apps

Real-Time frameworks: http://derbyjs.com/ , http://meteor.com/ , http://www.socketstream.org/

What are the best packages to use to build a JSON API backed by Postgres?

mb4, for small apps have a look at meteor.com. One of the newer frameworks sitting on top of Node.js. Have a look at the screen cast for the party application. Your jaw will drop.

@DonNguyen I’ll check it out

Meteor is amazing, although I would treat it as a separate entity to Node

it’s built on top of Node, but abstracts it away from you

i.e. you can’t install arbitrary packages from npm

markdalgesh, agreed. However for smaller single page web applications, I believe Meteor really shines. It is as you say somewhat of a different animal

Meteor has its own package manager, i think

@DonNguyen It is pretty amazing, I built a tool at work using Meteor for real-time communication between offices

mb4, I have used https://github.com/brianc/node-postgres with no problems. Plain vanilla Express.js is fine for serving JSON

@suniled Yeah, it has “Smart Packages”

Does anyone have Node.js vs. Other web server performance test? How can Node.js serve under heavy traffic for an app?

@nokrosis I tried once to do some benchmarking https://github.com/gvnn/nodenight-benchmarks

but it’s not a “REAL” benchmark, it’s just a fibonacci calculation

but it gives you an idea… and php is surprisingly fast

What would you guys like Node to support that it doesn’t at the moment? What are the pain points?

@Nokrosis for what its worth http://zgadzaj.com/benchmarking-nodejs-basic-performance-tests-against-apache-php

I think it’s worth mentioning that a choice for or against Node should be based on what you’re trying to achieve, not so much its performance

Nokrosis, benchmarking is a very tricky topic and it very much depends on the exact test you’re doing. However as a very broad generalization, it generally benchmarks favorably compared to other platforms. Performance is one of its core goals.

That’s not to say that it’s not fast (it is), but for certain kinds of apps it’s a much better solution

If I want to write a real-time web app, Node.js is an obvious choice

I think that’s what I couldn’t get my head round initially. It’s suitable for some apps, not others.

also https://vertxproject.wordpress.com/2012/05/09/vert-x-vs-node-js-simple-http-benchmarks/

Nokrosis, in addition to what markdalgesh said, it is generally better suited for anything IO intensive (database, network, disk)

Anybody a fan of Meteor/Derby?

I’m a big fan of Meteor

suniled, that’s a fairly controversial benchmark I would recommend reading all of the responses as well in particular anything by Isaac Schlueter

I think there’s a bright future for the kind of approach that Meteor takes

yep… agreed

@DonNguyen I’m running a media site with heavy traffic, i’m saving posts as static html files and including them with php into the page layout. Can i use Node.js to run this faster with fewer server resources?

mb4, the package system is pretty amazing it supports a lot of esoteric software and platforms. One of the pain points is the immaturity of the package system. Because anybody can release a package you’re never sure what you’re getting in terms of quality.

@markdalgleish what would you say are the top two reasons you like it?

@TJKoury First, how easy it makes it to write a single page app with automatic data binding to the server. Nothing else comes close.

@TJKoury Secondly, how it shares code between server and client by default, something that normally requires a bit of work

@mb4 i guess that the best practise is always to choose packages that are constantly maintained… a package that its last push was 1 year ago probably is not the right choice

Nokrosis, yes but I would say this is more an architectural thing rather than a platform thing. Without knowing the specifics, at a first pass I would say that using an indexed database would be much faster than saving as static HTML files

@markdalgleish thanks

@markdalgleish @TJKoury Model-View data binding http://blog.derbyjs.com/2012/04/14/our-take-on-derby-vs-meteor/

@DonNguyen I’ll try that solution with Node.js and an indexed MongoDB

towards the middle they talk about these ‘bindings’

Nokrosis, I would recommend doing a few small scale benchmarks first to help you decide whether the performance difference is worth the cost of doing a full migration across.

@suniled That article touches a lot on what I would say is one of Meteor’s biggest strengths and weaknesses

which is that Meteor have created their own ecosystem

They’ve made it ridiculously easy to create an app and deploy it to meteor.com

I don’t have any experience in meteor or derby but I’ve read a lot about them and some of my colleagues use meteor and others derby. It seems they both solve the same problem but with just a few tweaks that are different

But of course, that comes at the cost of separating itself from the Node community and all of its existing modules

but wouldn’t you say the down side is that ‘its there own ecosystem’

@markdalgleish Thanks again. I’ve looked at different ‘end-to-end’ solutions in the past, but always ended up writing my own b/c it seems they aren’t mature enough.

@suniled It depends who you are

@markdalgleish plus… i believe meteor got ridiculous amount of funding to get off the ground

@suniled something like 11 million

…kind of makes you wonder what their business plan is going to be…

from stockoverflow – http://stackoverflow.com/questions/10374113/meteor-vs-derbyjs

*stack

I can’t imagine they’d be making bank off of hosting alone

TJKoury they are going to release an enterprise product called Galaxy

Oh, ok.

@TJKoury hosting meteor apps

to be honest I’m not a big fan of these frameworks, I’d rather build a set of apis and then a front end application that consumes these apis

keep the 2 layers separated gives me the chance of change the backend layer later on, keeping the same front end

Hi and welcome to those of you that have just joined us. There is about 15 minutes left in the session, so if you have questions, make sure you get them in soon. The team of experts today are the guys behind the first in our upcoming Jump Start series, which is about Node.js https://www.sitepoint.com/get-a-jump-start-on-web-development-and-design/

@gvnn Agreed

@gvnn hear hear

@gvnn, I don’t like all encompassing frameworks that absract the client / server divide either.

*here *here

@gvnn That said, I think things like Meteor and Derby are a sign of things to come

@markdalgleish I think Meteor/Derby will be the new PHP

What are your twitter / github handles ?

how does node.js fit into the WebRTC tech?

@markdalgleish you are right, that is what helps a framework to be adopted by a larger audience

but, by the end of the day, all you need is HTTP :p

@mb4 Mine is, appropriately enough, markdalgleish

would node.js fit nicely into a browser?

@suniled Maybe into ChromeOS…

@gvnn Yeah, Meteor makes things that are normally quite tricky in Node ridiculously easy, so a lot of people will be attracted to it

As always, the downside is when you want to do something that isn’t quite the “Meteor way”

@markdalgleish i think devs are a little hesitant in getting into meteor because of the ecosystem… it maybe dead simple to get an app running but then what?

mb4, my GitHub is nodeninja. Unfortunately with Twitter I’m still stuck in the 90’s

@suniled There are a lot of different kinds of devs out there. Some devs just want something easy and powerful, that does a lot of work for them.

Of course, that kind of power comes at a cost, which a lot of other devs don’t want

@suniled what you can do is to build your core as a module and keep it separated as much as possible from meteor

that gives you the ability to change framework later on

@markdalgleish @gvnn interesting

We have just over 5 minutes left in the session. Does anyone have a question that they haven’t received a satisfactory answer for?

i just have a private project in ruby/sinatra/nokogiri(mechanize): login into website, get html – and give back mobile html (iui) – can i use the backend part (login, parsing html, response like a api with json) with node? on client side then i can consume this json api with the client libraries you mentioned in the above links

slides. http://www.slideshare.net/studgeek/an-overview-of-derbyjs-and-meteorjs-for-the-nova-nodejs-meetup

@suniled what you can do is also run your own npm server :p

@gvnn …using a Metor app on their server…

I have learnt a lot here and the links posted have been a great insight, thanks!

muescha, that project will fit very well with Node.js. Anything relating to JSON consumption will fit like the proverbial glove

@muescha Yes, that kind of setup is definitely possible

@gvnn fun!

This has been great, thanks everyone!

@TJKoury could work… even though i’d suggest http://nodejitsu.com/ for hosting

@TJKoury Meteor allows you to generate a tarball with everything needed to host it yourself

@gvnn thats where i’m working on getting my first ‘coming out the closet’ site !

@gvnn I was meta-kidding, but good suggestion

@gvnn, where can I stalk you?

No worries. Thanks everyone for joining us this morning. Don’t forget that you can check out the Node.js book shortly. If you want notification when it is released, you can sign up here https://www.sitepoint.com/get-a-jump-start-on-web-development-and-design/

@TJKoury everything is possible… we can run a web server in a quadcopter device flying in your room

If you want to be notified of future sessions like this, you can sign up here https://www.facebook.com/sitepoint/app_115462065200508

@mb4 gvnnrules on twitter

@markdalgleish I did see that in the docs, but let’s be honest here, once you’re 6 months into a project the last thing you want to do is export all your data and re-ingest, while dealing with DNS registration

Feel free to hang around and chat for as long as you like – I’ll leave the room open

And I’ll be posting a transcript up on both sitepoint.com and jspro.com later today

@DonNguyen @markdalgleish mhhh the html parsing makes me some headaches. or can i parse a html page like jquery oder DOMDocument with css/xpath ?

Thanks @HAWK :)

One thing about Meteor is security. By default anyone can update anything right from their web browser. This is the default setting and you actually need to turn it off for security. One little gotcha.

@gvnn lol…buy me a quadcopter and I’ll mount a raspberryPI on it!

Thanks very much to Don, Mark and Giovanni for your time this morning – esp at 7am!

lol, plenty of day left for lots of node dev now then ;)

@tjKoury there is a whole podcast on Nodeup about robots, and the node comunity is going crazy

@gvnn holy crap, how have I not seen this? Thanks for the tip.

@DonNguyen Yes, the famous Meteor demo video is only possible because security is off by default

ps… for those that want to know more about node (and their deities) http://nodeup.com/

Thanks for the chat, I’ve been reading posts and following links, even if I neven used node.js before..! Really interesting, I’d sure give it a try!

Thanks Sarah. Thanks to everyone for tuning in and some great questions

never*

Thanks for the chat everyone, found it very useful.

Thanks all, node is webscale.

Thanks

muescha, I may have misunderstood your question but there are HTML parsing libraries in the package ecosystem that make this job pretty easy

If you haven’t tried Node before, at least do this today:
1) Install Node.js
2) Create “helloworld.js” with 1 line: console.log(“Hello world”);
3) From the terminal, run “node helloworld”

@DonNguyen thx (parsing was only one part of it)

mb4, from memory MongoDB is webscale, Node.js is rock star tech :-)

@TJKoury http://nodecopter.com/ ;)

muescha, what’s the other part of your problem?

@gvnn I just added myself to the mailing list!

Thanks all, had a blast.

all right guys, thanks to everybody… duty calls, time to go to work (the day’s just started in Melbourne)

Thanks everyone, hope I was at least somewhat helpful

Thanks for the tips!

Thanks everyone!

If anyone wants to ask me anything else, catch me on Twitter

My handle is @markdalgleish

@gvnn time to go home here!

Thanks guys. I will be around for a little bit longer in case we missed anything.

certainly gives an interesting insight into node usage :)

and a lot of ideas

someone needs to work out how to double the hours available per day ;)

i think parsing is only the “problem” – (a) login to website (b) get html (c) pase html (d) reponse with json / something like an external api-proxy – add a api to a website (from extern)

I also wrote this blog article recently that some of you may find helpful getting started with creating Node modules:

markdalgleish.com/2012/09/test-driven-node-js-development-with-grunt/

@DonNguyen i think parsing is only the “problem” – (a) login to website (b) get html (c) pase html (d) reponse with json / something like an external api-proxy – add a api to a website (from extern)

Duty calls, I’m off to work now – it’s 8:06am here in Melbourne

Thanks guys

Thanks Mark.

muescha, all of those steps have been properly battle tested in Node.js you should have no problems doing all those things

cheers Mark

@DonNguyen i was thinking about to move from sinatra to rails – but i think node.js can be an other idea – i like to be the response from node act as an api

I have to head off too guys. Thanks for joining us this morning.

It’s the kind of job that Node.js was meant to do

thanks Sarah

@DonNguyen you have some keywords / packages which are interesting for this to look into it deeper?

https://github.com/tmpvar/jsdom for parsing

core http module should handle logins, Express.json for serving JSON

thx a lot :-)

you’re welcome