Canvas
      * Bitmapped graphics
      * Immediate mode
      + Fast for simple things
      + Minimal drawing overhead
      - Interactivity has to be handled manually (e.g. by checking whether click coordinates coincide with a drawn button)
      - Animating things (usually) mean redrawing most of the canvas for every frame
      + Ability to manipulate pixels if needed
      * Everything is drawn manually

    SVG
      * Vector graphics
      * Retained mode
      + Fast for simple things
      - Everything that needs to be drawn has to exist in the DOM
      + Interactivity can be handled via events by the browser on every element if needed
      + Animating things (usually) just mean to manipulate that thing in the DOM
      - Limited ways of arbitrarily manipulating the composed result
      * Everything is drawn by the browser
      + CSS support
      + Sophisticated text rendering support

It's mostly a trade-off of what you want to do. The things marked with * above are not clear benefits or drawbacks.

For a chart that consists of axes, ticks, a plot and maybe a few other things I'd always go for SVG unless compatibility constraints force me to go to bitmap graphics. It just maps much better to vectors than to pixels, but there are limits, of course. Heatmaps often are better drawn via canvas, especially those where there are no discrete shapes but rather blurry blobs of colour.

Performance is a difficult topic and hard to compare, beyond simple things: In canvas (mostly) your only option of optimisation is to draw less, i.e. touch fewer pixels; with SVG being rendered and composited by the browser it often boils down to »don't move elements that don't have to move« to enable the browser to render less and not throw away every partially composited texture. In IE and Chrome SVG has quite great performance, in Firefox you have to be careful what you do. Uusally it scales well to a few hundred or thousand DOM elements, though, depending on what you do.

Canvas is faster, as it basically is just a bitmap, while an SVG chart with a million points would correspond to at least a million DOM elements.

3 years ago, I benchmarks it with thousands of lines: SVG was faster than Canvas in Internet Explorer 10, probably due their extensive long history with their VML legacy rendering. But SVG was a lot slower in Chrome and Firefox than Canvas probably due graphic card accelerated Canvas implementation. SVG wasn't supported in Android 2.3.x. From my experience back then working directly with raw-SVG in HTML5 wasn't a great experience SVG is based on XML after all. Working with raw-Canvas meant implementing a basic render loop myself which refreshes the area and the Text related API is a bit complicated and slow.

SVG performance in IE (since IE9) has really been and is stellar, indeed. Chrome caught up mostly by now and at least at work the difference between Chrome and IE for certain tasks is that Chrome spends more time rendering and IE spends more time JavaScripting. Firefox SVG performance has been disappointing, mostly and mobile browsers are often better off with canvas as well since performance is generally constrained there.

SVG performance in IE doesn't date back to VML, though. They just made hardware-accelerated rendering a major feature of their browser back then, while no one else was doing it properly. Similar to how Chrome started with JITting JavaScript to make it fast and everyone else caught up on that afterwards.

You don't necessarily need a million DOM elements. Data points could mean points of a single plot, in which case you have a single path element with lots of points. Generally you can almost always bunch things together in a single path to reduce the number of DOM elements (with other tradeoffs, of course).