You heard me. He isn't, plain and simple. This opinion which I've heard thrown around quite a few times is still false, and it will be unless the NDP have a breakthrough next election.
Let's review the two leaders, shall we?
Ed Broadbent was NDP leader from 1975 to 1989, going through four elections (1979, 1980, 1984, 1988), and was by far the party's most successful leader. His first election he managed to take his party to their second highest vote totals in their history (17.88%), and in the next three elections he managed to get a greater share of the popular vote than Layton has so far managed. Here's his results:
1979 - 26 seats (9.22% of all seats) @ 17.88% of the popular vote
1980 - 32 seats (11.35% of all seats) @ 19.67% of the popular vote
1984 - 30 seats (10.64% of all seats) @ 18.81% of the popular vote
1988 - 43 seats (14.58% of all seats) @ 20.38% of the popular vote
Jack Layton has been NDP leader since 2003, and he's so far gone through three elections (2004, 2006, 2008), and will probably go through a fourth. Here's his results so far:
2004 - 19 seats (6.17% of all seats) @ 15.68% of the popular vote
2006 - 29 seats (9.42% of all seats) @ 17.48% of the popular vote
2008 - 37 seats (12.01% of all seats) @ 18.13% of the popular vote
Now, here's a fun mathematical trick: calculating vote efficiency. Basically like how you'd calculate fuel efficiency - work performed/energy expended, though in this case, it's seat results/popular vote results. For example, in 2008, the Liberal Party's popular vote was 95% efficient - or 25% of seats (77) / 26.26% of the vote. It's a fun way to see in a FPTP system how much of your vote actually works for you. The Conservatives had 123% efficiency (eep).
Anyways, the below line graph shows the two leader's vote efficiency over their elections (fyi, Layton's is scrunched up for comparative purposes).
Indeed, average it out, and Broadbent's NDP managed to have 59% vote efficiency during his tenure, while Layton's NDP has has 53% vote efficiency. Even if you discount 1988 for Broadbent, it's still 55-53. Hm.
Furthermore, and I really, really wish I had data from the era to show you, but everyone knows that Broadbent regularly surpassed both the Tories and the Liberals in the popular vote. Layton, as far as I can see, has yet to surpass the very weak Liberals, and at best managed a tie. Not exactly inspiring.
And I know someone will say that Layton has managed to be nearly as good as Broadbent despite a more conflicted electorate, and is therefore more successful. First, let me point out that is a very subjective opinion. Secondly, let's also note that even if it was true, Layton come near Broadbent's score in many areas despite more parties existing, but his vote share is still more inefficient.
The only areas where Layton can really claim to be superior is in some regional areas, specifically Ontario, where it really counts. But more traditional areas of NDP support, such as BC, MN/SK, and Atlantic Canada, he falls behind Broadbent, or in the latter case, McDonough. Take that as you will.
So, once again: Jack Layton is not more successful than Ed Broadbent. If it changes in the next election, then I'll gladly admit it (though I severely doubt he will). Until then, however, the fact remains.