really, nothing here

Announcing Freerisk

2009-03-18T16:56:00.004-06:00

Last week, Toby Segaran and I announced our new project Freerisk at the O'Reilly Etech conference, in a talk entitled "I Just Don't Trust You: How the Tech Community Can Reinvent Risk Ratings".

In a nutshell, we're building a semantic database for all credit risk related data that we can find, and offering an API for others to implement their own credit risk calculators and publish these ratings back into the system.

This will allow investors to collect a diverse set of credit ratings and select the ones most relevant to them, make it easier for experimenters to create new risk metrics, and avoids a number of the payment for service problems in the current credit rating industry.

There's more to come, and I'll share those announcements here.

For now, here's our talk at Etech

For those of you that understand the current problems in the credit rating industry, you may want to skip forward to about the half-way point where we present the solution.

this is old.

2008-10-06T23:04:00.000-05:00

but oh so good.

Hadoop Reduce Value-Iterators are Flyweights

2008-09-28T18:20:00.006-05:00

Hadoop reduce iterators are pretty broken, so realize that before you waste tons of time like I did. The following won't work (never mind the crap job I'm doing protecting my heap):


List entries = new ArrayList();
while(values.hasNext()) {
 entries.add(values.next());
}

If fails because Entry is a flyweight instance that gets recycled on each iteration. A nice optimization if you need it, but a rather heavy-handed assumption and one that should have been documentated, and an optimization that in this case leaves you with a List populated with references to the same object instance. This could take you a while to track down.

If you, like me, can afford to cache your values in your reduce step and have a multi-pass algorithm which requires you to cache do something like this:


List entries = new ArrayList();
while(values.hasNext()) {
 entries.add(new Entry(values.next()));
}

where Entry(entry) is a copy-constructor (you could use clone() if you've done nothing sexy to your class and for some reason hate portable design standards).

Hadoop is a great project, but they really drop the ball on documenting oddities like this.

Podcaster-gate

2008-09-20T12:01:00.002-05:00

I'm impressed by the anger directed at Apple for Podcaster-gate. Good community backlash is always healthy. If you haven't heard about all this you can read more about it here to catch up, but in short, Apple didn't allow an application that competed with their iTunes application onto the iPhone, even though it didn't violate any of the provisions of the license agreement.

However, I'm quite unclear, why anyone professes shock or surprise at the situation.

It shouldn't be surprising that this happened, though there are many reasons for consumers to be upset by this. I'm sure Podcaster is a wonderful product that does many things that can't be done yet on the iPhone. We'll all miss out on those. But Podcaster wasn't ever going to make much money on this or ever reach scale because the strategy was naive and flawed; and so in the end, Apple did it's developer, Alex Sokiryansky, a favor.

Don't try to play nice with a bully

The strategy was naive because it assumed that Apple wouldn't be petty and protective of its turf. Past performance is no guarantee of future results but with Apple it's pretty a good estimate. Apple has a long history of screwing up independent software vendors' businesses and products by replacing them with built-in features. Konfabulator (now Yahoo! Widgets) and Watson are shining examples of wonderful, well-loved, products still trying to find the knife Apple stabbed them in the back with. Assuming your product will be treated differently isn't rational. The very fact that Apple wanted control over the App Store should have caused the cautious to flee the platform as fast as possible.

Don't take on your platform providers when they write the rules

Furthermore, the strategy was flawed because Apple had to react aggressively towards this product--Apple isn't just in the business of producing iPhones, iPods, and computers; they are in the business of owning the distribution of video and audio content, which lets them command premium pricing on those products (yes, yes, the design is a differentiator, too). Putting a product like this on their platform threatens the network effect that Apple is trying to build between their products--it threatens almost everything that has differentiated Apple circa 1997 from Apple circa 2008. These are the network effects that keep us coming back to them to buy their products--and cause us to deal with lesser products like Apple TV. Apple can't allow simple content distribution on their products because it can't allow the value they dominate to go away. This may or may not be best for consumers, but it's certainly best for Apple.

Don't assume you and your platform provider have the same goals

Finally, within the community, there's an argument that the process of accepting applications after they're done is problematic because independent software vendors cannot know ahead of time if their product will be placed in the App Store, forcing them to invest with no guarantee of an ROI. That is, of course, true. Because of this uncertainty, companies will need to work on products with a larger payoff than before, earning the higher risk these products take on. But the independent developer's product is Apple's good fortune. Who wouldn't want the natural quality filter that these products carry? Quality, not quantity, is a very Apple-like notion that this policy underscores precisely. If you don't help tell that story, they won't help you sell your product. We learned that from this fiasco.

Don't confuse forums for throwing sheep* with real platforms

There's also a question of what type of platform this is: are we looking at a platform designed to create real value for users, or are we supposed to building sheep throwing applications for it, to increase stickiness and virality for Apple. Despite assertions from Apple, it may be, and appears to be so far, that the iPhone will never be a platform to base your company on, but rather a platform in which you can distribute complimentary products to your core offering; products that grow your own strength while not threatening, or in fact complementing, Apple's own offerings. There's substantial financial evidence of this. Apple's iFund for creating new companies around iPhone applications is deploying capital at a slower rate than many expected, yielding only five investments so far, each of which is looking to harness the iPhone to access value from a market Apple doesn't have a hand in.

Net lesson: Apple owns this platform, and you should act like a good guest wanting another invitation, rather than than trying to grab all the attention for yourself.

The good news is there's a pretty interesting new platform coming out shortly that should be more than happy to host Podcaster.

* Tim O'Reilly first used this pejorative at last week's Web 2.0 Expo keynote.

Perhaps Denmark shouldn't outsource it's cultural exchange to these jokers

2008-09-20T11:57:00.004-05:00

But I'm available for the right price.

Ostensibly a tribute to Olafur Eliasson.

Contains expletive laden substance.

Hat tip the quiet quiet

blogging at O'Reilly Radar for a bit

2008-06-29T19:57:00.003-05:00

I'll be guest blogging on the O'Reilly Radar blog for a little while.

You can see my first post here.

Is language a function of your networking?

2008-06-10T22:22:00.003-05:00

David Gravel is quoted as thinking so given the massive efficiency to shared languages in a global world. It's a tidy and nice argument.

If you picked two people at random off the face of the earth and asked them to pick one language in which to communicate with someone they knew nothing about, which language would each person choose? The language they’d pick would depend on a series of “reciprocal expectations” — best guesses not just about which language you suppose the other person speaks but which language he thinks you suppose he will speak — which depends, in turn, on which one you think he thinks you suppose he will speak. And so on, until your head swims.

In today’s globalizing world, the probability is increasing that two random people would choose English for their best chance at unplanned linguistic coordination. And this isn’t merely a thought experiment: it’s being played out, with more information among the parties, in the decisions of hundreds of millions of people now learning English as a second language.

It reminds me a bit of David Foster Wallace's wonderful argument that grammar, too, is a function of your social setting (the pre-SNA way of saying networking).

You'll need a Harper's Subscription to view the link, but you should have one of those anyway.

These arguments make me wonder if, given this powerful network effect protecting the English language, there's any need to mandate a single language in the U.S., Britain. I'm also wondering just how the Quebecois intend to maintain French over time.

Indian Firms, Where Innovation Goes to Die

2008-06-10T00:00:00.003-05:00

The Navi Radjou at the Harvard Business Blog writes about the dearth of innovation going on in India. It's a structure problem according to the analysis. The culprit works out to be family-run, risk adverse firms with nepotistic power structures.

No doubt.

The solution: bring on "Web 2.0" software to break apart the power structure and liberate the wisdom of crowds.

No doubt, again, except....

...it seems to me that the real culprit is the "family-run, risk adverse" part. These power structures form when shareholders have few legally enforceable rights, so power is only transferred from an owner to trusted family agents, regardless of skill. So you, the clever underling will never get at taking the reins of the company, lessening your desire to take on the risk of dramatically changing the direction of a firm. (The counter point firm, Nokia, is not a "family-run" firm).

All the software in the world can't release innovation without incentives on the part of your clever workers, and in fact, the lack of leadership opportunities likely creates pressures on clever employees to take their ideas outside of the firm. (It's interesting to note that overall innovation is strong in India, just not within established firms - with venture investments in 2007 up 166% over 2006)

My takeaway - software is great, but culture and structure are far more important determinants of entrepreneurial success in a firm. Firmer rule of law would free up the family-restricted transfers of power, and open up new incentives for Indian workers to donate their ideas to their firms.

when mbas go wrong

2008-06-05T01:32:00.001-05:00

On a related note, my technology strategy professor laughed at the idea that Apple might have a competive advantage in "style and taste".

apparently i could be more interesting

2008-04-18T15:57:00.004-05:00

Worth noting is that the bounce rate on "jesper andersen" is 75%. They're searching for a different one. Great news for my friend Rob and his company Knowledge Bid, though.

in which i get too snarky by half on the web

2008-04-07T19:10:00.006-05:00

Last night I got up nervous about my interview for yet another Kauffman Fellow position, this time up in Canada.

To kill some time I cleaned out my rss reader and caught an interesting post by Fred Wilson here, regarding a rather poor (I believe) HBS post here.

I sleepily banged out a snarky message regarding what I thought of the HBS posting and posted it to twitter. It read:

apologies to @fredwilson but does anyone understand the economics of this post? http://tinyurl.com/333y9c seems naive, lacking perspective

By which I was referring to the HBS post. I think that got lost because when I arrived in Toronto I got these tweets:

@jandersen the way I use my blog is to 'think outloud' and have other correct me. It works like a charm.
@jandersen the post may be naive but the comments are great

Whoops. That wasn't what I meant. But at least someone was listening.

What I actually think is that Fred pulls out the only salient points in the HBS post and draws out an interesting forward looking thesis. I'm just not clear on what in the HBS article warrants all that attention. I'm of the opinion that the post isn't novel or particularly correct.

It's not novel in that its essentially a restatement of creative destruction. Things work until profits get competed away and you exist because you make just enough to exceed your liquidation value. It's a common phenomena that has nothing to with creating competitive advantage and everything to do with losing it. The industries that Umair Haque mentions are very mature industries that have undergone economically predictable consolidation and margin reduction phases that have nothing to do with their strategic choices and everything to do with competitive entry.

It's not correct because there's almost nothing you could do over a 100 year period to maintain the sort of margins that avoid the situation he's describing. It's inevitable. You don't have IP protection, other industries are churning to create entrant opportunities, and, frankly, its far too long of a timespan for anyone to care about anyway.

It's entirely possible that technology is speeding things up a bit, but even so, I'm not sure that it's the same effect.

In a nutshell, Fred's right, there's something disruptive going on and there's very likely to be some interesting investment theses there, but it seems to me to be a natural business cycle disruption, and not a new movement. This seems like its going to work out a lot like the oil shock effects in the very early 80's begetting many numerous startups, and it's up to someone to make the next Staples that will cater to all of their needs.

Also, for the record, Fred's commenters are smart. No doubt.

Twitter's growing faster than you think

2008-03-28T08:43:00.001-06:00

I recently told my friend Lane Becker that I didn't think Twitter was growing much anymore. I was wrong

Twitter's growing at a very healthy rate. And in this case, the compete numbers, even if absolutely correct, don't tell the whole story. The first thing any serious twitter user does is download a desktop client, or start using TwitterIm or Hahlo and being relying on the Twitter API, whose usage isn't reflected in Compete statistics. So the Twitter growth is really a measure of some baseline passive users, and the flux of new users entering the system, which means that Twitter's attracting a linear number of users each month, or better.

Congrats to Twitter. Now can someone please explain what Twitter is to all my friends, because I'm failing at it.

SNMP for feeds

2008-03-23T14:22:00.004-06:00

Josh Kopelman at First Round wants an "SNMP Dashboard" for feeds. I for one wholeheartedly agreed. Someone needs to help me managed my 500 feeds.

I'm a data junkie, and probably a two-standard deviation edge case for now, but like Josh, I think this will end, and I'll be normal shortly. I think you can see these problems in microcosm on Tumblr right now. Content comes in either as a river of data, with no re-prioritization, no de-duplication, and no categorization. It's overwhelming, but the only other choice is to avoid content with great care. This is an unattractive solution.

Josh is correct, too. These aren't technical challenges. Heck, we had them solved, more or less, at Screaming Media in 2000. There's a degree of fit and finish that makes this project hard. Content selection and prioritization are tricky items to do in a trustworthy way (as opposed to statistically relevant way), and when there's no ROI statement to be made, consumers might reject a service that didn't take great care in presenting these choices to the user. But that's likely solvable.

What's really interesting is how the market is working on a different, and I feel temporary problem. Most projects are focused on discovering new feeds (which is, good lord, not a problem I have), Google Reader, Findory (deadpooled), Friend Feed, and Persai are all devoted to this temporary stop gap before we reach a world in which we live and breath feeds and syndication.

It's roughly the same engineering problems, with a more difficult design problem coupled to it. Hopefully one of these, or a new service will come in to fill this whole.

Angels vs. Professional Investors

2008-03-22T11:03:00.003-06:00

Research suggests that you'd be foolish to accept both angel and professional investment money for your startup. Sounds like bunk to me, but others are eating it up. The paper does have a fair number of controls in it, but the conclusions still seem a bit aggressive to me.

Net summary: Having more investors involved complicates things, and angels and professionals often have trouble coinciding their mutal interests. Fair enough.

My take home: this is more evidence that if you ever have to raise money for anything, be sure that you are getting more than money with it, and that you and your project's needs match your investors interests; try to make sure you bought key help for your next milestones for all that equity in your project. Most likely you'll have improved your chances of success if you do that.

Via [Paul Kedrosky]

Watch walmart take over the world (well country)

2008-03-20T18:11:00.004-06:00

Friend and "data magnate" of Metaweb Toby Segaran, just published an animation he made of the growth of Walmart over time, built exclusively from publicly available data joined together using the psuedo-typed join algorithms they own there.

You can see it here.

Toby's promised more improvements and I'm excited to see what else he can come up with.

Note: Edited for fat fingered typing problems.

Why facebook apps are important even though they annoy the crap out of me

2008-03-18T22:51:00.000-06:00

Facebook applications are here to stay, and are, I suspect the key data point that will provide Facebook with an edge in the social advertising space. The reason being is that the applications give Facebook a unique way to look into how closely user are located on the social grapb and in what manner they are close on the graph. (Well, technically, it actually create a set of graphs, one per communication channel that can be projected down onto a single graph using from matrix transform, but whatevs.) By combining this new data, with the permission graph that all the social networking systems have (this is what you are used to, so-and-so can see this data but not that data, etc) you start to get a pretty persuasive package of socially intelligence agents with permission settings that allow access to and from the data with user confidence.

The genius here is that by forcing all of this networoked behavior over their permission graph, Facebook makes sure that the implicit graph and the explicit graph overlap, an furthermore, the implicit graph is a subset of the explicit graph, guaranteeing that for any implicit edge, there's a explicit trust or permission edge dictating what inferences can be used from that edge. Genius. Contrast that with OpenSocial, which gies google a myriad of implicit edges, but no explicit edges, and even those edges that it can find will exist in a federated system of graphs that are, most likely, highly degenerate. What a mess. It will take Google a long time to sort this out and come up with a way to unlock the implicit behavior for their clients. Meanwhile Facebooks new and rather impressive R&D team will be churning out one novel application of implicit networks after another.

Now if only someone could give them an applications that generated an implicit network based on something other than zombies, vampires or ninjas.

Why I can't go on vacation

2008-03-17T11:36:00.005-06:00

These pictures are so gorgeous, that I want to go there, sub zero temps and all. As far as I'm concerned these relics are our modern marvels of the world.

Edward Burtynskey does this obsession better, and with more philosophical and theoretical flair, I just react to these photos emotionally. There's a lovely U.S. Navy graveyard in the Suisun Bay near San Francisco that's fine, but the primal nature of the russian ships, ripped apart by the ice, is far more moving to me.

Via [IOC]

Tyler Cowen

2008-02-27T13:45:00.003-06:00

Tyler Cowen asks today, why not increase the penalties on insurance shirking? To which, I reflexively replied, "agency problems", patting myself on the back for giving the one answer in economics that's almost always true -- readers, you are dealing with a deep thinker here.

But couldn't one eliminate the agency problems in most tort litigation by only handing over damage fees to the victim, and returning punitive penalties to the government, for distribution to society at large -- since presumably the punitive damages represent a tax to be charged to a serial infringer who gets caught infrequently? What's wrong with this idea?

Anthem for Obama

2008-02-02T14:26:00.000-06:00

I've kept my politics out of this blog, because, well, it's not something I'm particularly fond of sharing. But how many candidates can generate the kind of excitement during primary season that Obama has?

Will.i.am makes an unsolicited video inspired by and featuring Obama's Yes We Can speach

Shepard Fairey starts his own viral campaign for Obama lending his iconic style to a poster campaign for the candidate and now also a "Obama has a posse!" style paster (sticker) campaign. http://obeygiant.com/post/obama-pasters-update

Make your own choice on Tuesday (or whenever) for whatever party you belong to. But do consider the passion that Barack Obama gives people when making your choice.

Congrats to Acquia

2007-12-20T06:08:00.001-06:00

On my way home from Foo Camp last summer I was lucky enough to get a ride back with Dries Buytaert and others. The entire trip home was spent discussing how and when Dries could take Drupal commercial, and the various business models that could support the Drupal development effort. Like other Open Source projects the issues are tricky and subtle but Dries seems to have solved those problems and announced today a MONSTER $7M round of funding, from among others, O'Reilly AlphaTech Ventures for his new firm Acquia, designed to capture to commercial value of the Drupal project.

Dries was impressive at Foo Camp and I'm excited to see what plans he has for Drupal, both in the near term and the long term.

Chicago Ignite

2007-12-05T18:19:00.001-06:00

If you read this blog, and live in or near Chicago, then you'll want to be at the Ignite- Chicago I'm hosting with Sean Harper tomorrow. You can read more about it on our blog: http://ignite-chicago.org/ or on upcoming: http://upcoming.yahoo.com/event/309951/

We've got a really great list of speakers that I'm in awe of, and I think if you can brave the cold weather you'll have a good time.

Harper Reed

Jason Fried

Erin McKean

Keith Schacht - something about Facebook Apps or Google’s OpenSocial

Mark Roth - something about Local Media

Justin Kruger - Using your graphics processor for scientific computing or something on Prize Economics and innovation

Derek Wade - Scrum

Leon Chism

Brawley Reishman - Building your web application with offshore labor

Jason Rexilius

Matt McCall

Massimo DiPierro - Gluon Web Framework

Paul Caswell - Thinking Together

John Pasky - Protecting your IP in 5 minutes

Sean Blanton - Using Perl to Control Java

Ravi Shankar - “Innovating Process, Methods and Tools for the Web 2.0 World” or “AJAX Frameworks and Tool kits - When to use what, for building your Rich Internet application”

Knowledge Bid launches

2007-11-21T10:42:00.000-06:00

In an effort to become a serial entrepreneur as rapidly as possible Rob Webb has launched his new service, KnowledgeBid. KnowledgeBid is a forum to link people who want access to deep industry or technical information with those who have that informaiton, and is trying to close the gap on the 99% of the data that isn't avilable online. Providers of information will get paid an hourly rate for their time, and the conversation happens over a phone call that KnowledgeBid will set up for you.

I'm in there with expertises in data mining, social network analysis, and (HA) venture capital. Beat me out to get your own contract work.

You can also check out Rob's other website, Analyst's Edge, a digg for finance types.

Statistics for profit

2007-10-30T21:28:00.000-06:00

Joel Spolsky's Evidence Based Scheduling is a smart idea. Really smart.

I have a few questions about it I'd love to have answered, but there's no comments section or forums entry for this post:

Do you make normal assumptions for the error on estimates? Seems more likely that you would expect the distribution of incorrect estimates to skew towards too long?

Aren't the errors heteroscedatic, I'd expect the error to be larger on larger bites of data (and likewise, shorter on smaller bites)?

Can you produce a conditional distribution for the likely finish dates rather than just a confidence interval on the expected on?

Can you track dependencies in teh schedule, that is can we propogate an uncertainty from one milestone on one developer into the start of another milestone for a second developer, this would really help measure the interaction costs of a large team, which seems to be missing in this model?

Why is this technology stuck inside of FogBugs rather than being its own product. There are tons of use cases for this in all sorts of knowledge labor fields, why not enable those? Also isn't in an orthogonal feature set to defect tracking?

I don't need these features, but having them would really make me happy.

I like me some precious

2007-10-18T01:39:00.000-05:00

okay. i get it. you think Darjeeling is a bit precious.

Wes Anderson spends a lot of time on the art direction. And some of the shots are maybe a bit contrived.

But seriously, where else do you get to shear joy of the mundane parts of our lives. Name another movie that makes a fetish of luggage. (Though its also the obvious metaphor in the film)

A film about moving on in which the location moves along with you? On a pre-ordained route inscribed by steel rails and wood ties? You love that stuff. You know you do.

Wes Anderson remains films answer to Richard Powers. Forever constructed intricate and meticulously crafted structures in which no item has no place, and all the parts form the unique and necessary conditions to maintain the story, Anderson has a tendency, like Powers, to lose the soul of the story sometimes under all teh technical greatness. Just be grateful no movie is 800 pages long. Its also worth mentioning, that like Powers, Anderson might need to move on from handling the lives emotionally stunted and withdrawn men of roughly the same age as Anderson himself.

But none of that takes away from the joy of the games, puzzles and distractions his movies provide.

And did you catch that micro-movie on itunes, Hotel Chevalier? A masterpiece. Better than Darjeeling, frankly. But you can't claim all the visuals cues passed back and forth between the two didn't increase yr enjoyment of both movies.

The world has spoken and it judged Wes Anderson precious. Which I still don't get as a pejoritive so much. But I'd rather have more artists fall in love with the sound of them trying to make something than have then fall in love with the sound of their own voice.

Helvetica

2007-10-15T12:45:00.000-05:00

How do you make an awesome movie about a font?

You start with a great ambiant rock soundtrack

Then feature a 2 minute long montage of a goofy whitehaired man running around New York pointing at signs with Helvetica in it.
Then don't so much make it about a font, but about the development from modernism to post-modernism and the subsequent rejection of post-modernism. Tell a narrative about who we are as consumers and how packaging reflects our identity.

Helvetica is all of that and more. Bringing together interviews with what I understand are some of the most important designers I've never heard of and visually telling a history of design by focussing on such a small topic that anyone (even I) can get their head around it really does wonders.

For two hours JP, SB and others enjoyed this movie and everyone came out smiling.

Check out clips here and see if its for you. Listings on on their blog.