Skip to content

Open Source

Sonar in the news

Sonar - 10 hours 46 min ago

Welcome to the roundup of blog posts and pages that mentioned Sonar last month…

Measuring code quality with Sonar
By Allard Buijze, 26 February 2010

“At JTeam, we continuously strive for good quality code. The reason is very simple: bad quality code slows down the development process. The small investment pays out in even the simplest of projects.”

Sonar: a valuable tool for every Java shop
By Hans Westerbeek, 22 February 2010

“If your organization develops Java applications, chances are that Sonar could be very valuable for you. Sonar helps you calculate your technical debt by analyzing your project’s source code.”

Getting More Agile – 2009 in Review
By Thomas Ferris Nicolaisen, 21 February 2010

“With our mavenization efforts, we were recently able to make use of Sonar (which rocks, btw) instead of the old, more primitive (boring) Ant reports in Hudson.”

A Sonar Cobol Plugin
By SonarSource, 18 February 2010

Coming soon… “SonarSource has developed its own Cobol parser and packaged it in a Sonar Cobol analyzer. It allows to perform objective and automated Cobol code quality and standards reviews against pre-defined rule sets and coding best practices. It also generates standard code metrics like number of lines or density of comments”

Add code quality metrics to your GateIn Dashboard with Sonar
By Jeremi Joslin, 11 February 2010

“Sonar is an open source platform to manage code quality. It enables to collect, analyze and report metrics on source code. At eXo Platform, we use Sonar to manage and monitor the quality of our codebase.”

Sonar Integration for AnthillPro
By Eric Minick, 10 February 2010

“A Sonar plugin for AnthillPro is now available through Urbancode’s Supportal (support portal).”

Sonar SCM Activity Plugin version 0.1
By Evgeny Mandrikov, 27 January 2010

“People, who follow me on Twitter, already know about new plugin for Sonar developed by me. Meet – Sonar SCM Activity Plugin.”

Java Build Server
By Manuel Küblböck, 23 Januray 2010

“In my last Java project, I set up a build server with Continuous Integration (CI) capability. I am a big fan of Test Driven Development (TDD) and I quite enjoyed Hudson telling us right away when someone checked in code that broke the build. It just gives you so much more confidence in your code and keeps it releasable at all times. In addition, we used Sonar to measure the quality of our code.”

Categories: Open Source

Creation Stories

JUnit Max - Kent Beck - Sat, 03/06/2010 - 02:39

I’ve realized for some time how powerful creation stories are in spreading ideas. Think of an idea that has power in your life or work. Do you know the story of the origin of that idea? If you don’t, does the person who introduced you to the idea? My experience is that I know the story or the person I learned the idea from knows the story. An idea only spreads as far as the story of its creation.

When I’ve had impact on software development, there has always been a story attached:

  • The group of us on a hillside in Colorado experiencing patterns first hand
  • Erich and I writing JUnit on a plane going to OOPSLA
  • The C3 project at Chrysler that spawned Extreme Programming
  • The workshop at Snowbird where we (well, actually I was pretty sick and didn’t do any of the work) wrote the Agile Manifesto

I was looking at the landing page of a two-person startup today and having trouble understanding the concept they were promoting. Their prose was obviously carefully crafted, but it was all written with abstract nouns–”we turn conversations into knowledge”. It didn’t make sense to me concretely. I suspected that the product might be applicable to my work but I couldn’t tell for sure.

My suggestion to the team was that they put their creation story on the landing page. If I could read about the moment they had their cool idea, they would have given me two valuable gifts–the emotional energy to keep working at understanding their idea and the beginnings of an intellectual understanding as well. To create their product they needed an epiphany, to be faced with a seemingly insolvable dilemma and for the resolution of that dilemma to come to them. If they told that story and I saw myself in the dilemma they presented, I would be pretty certain I was a good candidate for their product and I would be willing to work harder to understand it.

I would like to validate the use of Creation Stories empirically. Do Creation Stories work better than other forms of content in converting readers to customers? Here’s the deal. I would like to work with up to three teams who sell primarily through the Internet. You must be measuring your current conversion rate. Together we will write your Creation Story and then A/B test it. You must pay me for my help, but as I can’t even speculate on the value of this exercise I will trust you to set the price when the experiment reaches a statistically significant conclusion at what my help was worth to you. Assuming this idea really works well I will be selling this as a premium service. Please contact me if you are interested.

Categories: Open Source

Add CI Build Stability to your Sonar Dashboard

Sonar - Wed, 03/03/2010 - 15:33

Sonar is known as being the open source platform to evaluate and report continuously on source code quality. Its basic role is to evaluate the code technical debt that slows down productivity. Of course, several factors can lead to a productivity slump and poor code quality is only one of them.

Another one is the effectiveness of the Continuous Integration process. CI practice is directly inspired by Lean Manufacturing practices and the main goal is to “Create a continuous process flow to bring problems to the surface … as quick as possible”. When the Continuous Integration flow fails, this is very good feedback to hear : “Hey guys, stop the line. You first need to fix this issue : compilation failure, unit tests failures…”. But if the CI flow fails too often this is also a bad news as lot of time is spent fixing the problem and not developing new features.

The Sonar Build Stability plugin has been developed by Evgeny Mandrikov to evaluate the stability and the effectiveness of this CI flow. It currently enables to answer two questions on a given period of time :

  • How long does it take to get valuable feedback ?
  • How often was the build broken in the given period ?

Once installed, this plugin is automatically launched by Sonar on every project as long as the Maven pom.xml file contains a “ciManagement” node or that CI engine URL is configured in the plugin. The following new widget is then displayed in the Sonar web interface :

The current version 1.0.1 supports Hudson and Bamboo CI engines. Next version might be extended to cover Teamcity and respond to a third question : “When the build is broken, how long does it take to fix it ?”

This plugin is definitely a great addition to evaluate how Lean we are.

Categories: Open Source

R2#15 released !

IceScrum - Thu, 02/25/2010 - 23:20

Hi everybody!

The IceScrum development team is glad to announce the immediate availability of IceScrum R2#15 version.
Get it now in the download section!

Note: If you were using a nighty build that already included the Sprint Burnup Chart prior to January 10, 2010, please make sure to export your projects before upgrading, otherwise you will lose all your Sprint Burnup measures.

Changes overview

Here is a short overview of the changes in this version. Please, see the corresponding release note (english, french) to get the complete list of changes.

Korean and German localization

IceScrum² R2#15 now introduces the German and Korean language, thanks to two contributors: Romain Chartier (German localization) and Yong-hee Lee (Korean localization).

Korean and German interface (click to enlarge)

Please note that in order to fully support the Korean localization, the database used with IceScrum² must support and use the UTF-8 encoding character. If you are using IceScrum with its embedded database, you have nothing to do. If you are using your own database, you have to make sure that the encoding characters used to store data is indeed UTF-8.

If you are already using an existing database not configured with UTF-8, and that you don’t plan to use non latin-based characters in your project (you don’t plan to store data using Asian or Russian characters for example), you can still use your old database without changes, but please note that if you insert non latin-based characters in a database that isn’t configured to support the encoding, they may be replaced by question marks (« ? ») or others random characters.

If you were using MySQL, when creating the database, set the default encoding to « utf8_general_ci ». There is also a small change to do in the configuration.properties file, described in the about Configuration FAQ entry.

Sprint Burnup Chart & Burndown improvement

IceScrum² now offers a new chart in the Sprint Backlog : The Burnup Chart. It compares the total number of tasks in a sprint to the number of tasks done every days. You can remove the week-ends from the chart by clicking on the calendar icon next to the « Previous » button on the upper right of the window.

The Sprint Burnup Chart

The Sprint Burndown Chart has also been improved to display every days of the sprint on its X axis instead of gradually updating the axis every days.

The Sprint Burndown Chart

Context Menu alternative

Most of the context menus available in previous version of IceScrum² have been replaced with a new kind of menu, embedded in every post-it notes. The menu is displayed when moving the mouse over its icon, or when right clicking on a post-it note.

Note that the context menu in the release plan still remain the same (when right clicking on a column), for the moment, at least.

The new embedded menu on the post-it notes

ScrumMaster actions on a project

A ScrumMaster can now mask/unmask a project. When a project is masked, it will no longer show up in the projects list to users that are not members of the project.

When a project is masked, closed to new members, or has disabled the « change roles » feature, an icon representing the setting will now be displayed next to the user role on the upper part of the window.

The Scrum Master actions

Search in the Product Backlog

IceScrum R2#15 introduces a search feature in the Product Backlog. The search feature will apply a filter on the visible stories depending on the keyword(s) you will type in the search input box. To cancel the search filter, just change the activated filter, or leave the search box empty.

IceScrum will search the keywords (separated with blankspaces) in the stories attributes: title, description, note, and associated feature. The results in upper case that are suggested when typing characters represents the features corresponding to the keyword(s).

The search feature in the Product Backlog

We wish you a good Scrum experience with IceScrum !

The IceScrum development team.

Please note that in order to fully support the Korean localization, the database used with IceScrum² must support the UTF-8 encoding character. If you are using IceScrum with its embedded database, you have nothing else to do. If you are using your own database, you have to make sure that the encoding characters used to store data is indeed UTF-8.
Categories: Open Source

Securing access to projects in Sonar

Sonar - Thu, 02/25/2010 - 15:04

When used out-of-the-box, Sonar is a code quality radiator accessible by everyone at anytime. Like for JIRA, Hudson, a post-it dashboard or any other piece of the development toolset transparency is a key success factor for adoption. So, by default in Sonar, anyone can access any project under continuous inspection and navigate through it.

But of course, there are situations where securing Sonar is necessary. Let’s imagine for 2 minutes a consulting company that does development for customers and wishes to allow those customers to follow their own projects in Sonar. Since the company has many customers, it is necessary that group of projects can be isolated to make sure each customer only has access to his own projects. Prior to Sonar 1.12, this was only possible by having one instance of Sonar per customer.

Since Sonar 1.12 there are services available in the web interface to handle this and to cover the following use cases :

  • Secure a Sonar instance by forcing login prior to access to any page
  • Make a given project non accessible to anonymous
  • Allow access to source code (Code Viewer) to a given set of people
  • Restrict access to a project to a given group of people
  • Define who can administer a project (setting exclusion patterns, tunning plugins configuration for that project, …)
  • Define who can administer a Sonar instance

All those use cases can be implemented through the Sonar web interface and will take effect immediately. The way security is handled in Sonar is pretty classic as the security policy is based on the following three concepts : user, group and role (global or by project). Let’s take the example of the “Project roles” page available at project level :
sonar-project-roles
Three roles are available at project level : Administrator, User and Code Viewer. Users and/or a groups of users can be associated to each of those roles to get the required permissions.

User and group can be first created through the “Users” and “Groups” services available in the administration configuration section. Here is the screenshot of the “Groups” service :
Sonar-groups
That was authorization, let’s now talk about authentication. By default, user authentication is done against the Sonar DB (user table) but an external authentication engine can also used : OpenLDAP, Microsoft Active Directory, Apache DS, Atlassian Crowd … Three identity plugins already exist : two open source LDAP Plugin, Crowd plugin and a commercial one Identity Plugin. They all use the public Sonar authentication extension point.

To conclude, it is possible since Sonar 1.12 to easily implement a robust enterprise security policy. Those new functionality have been done with no impact whatsoever on Sonar users who do not want to activate security and want to keep full transparency.

Categories: Open Source

Smalltalk: Welcome to the Balkans.

JUnit Max - Kent Beck - Thu, 02/11/2010 - 05:28

Never blog when you’re mad. This is an excellent rule that I’m breaking right here.

For fun and education I started a MongoDB driver in Smalltalk. I wanted to learn about a NoSQL database from the ground (-ish) up and potentially provide something useful.

Step one: get it running in VisualWorks. Not a problem. I’ve used VW for 15 years. Lots of details to get right, but I’m a programmer. Details are slops in which I wallow.

Step two: find a customer. Again, not a problem. Someone has an application they would like to have populate a MongoDB database for later processing by other tools. Problem: the server is in Squeak.

Step three: port to Squeak. The pain begins. Just getting the code out of VW was a pain. I discovered Monticello, which is a nice concept for source control, sort of a Smalltalk-y version of Git (in the sense that they are both distributed SC systems). VW interface to Monticello is broken. Discussion with author resolves the issue. Code has escaped VW.

Loading into Squeak is no problem, but the Squeak programming tools are just too painful to use. Look no further for a shining example of the tragedy of the commons. Design requires inhaling–adding features, trying experiments, tweaking, going off in crazy directions–but it also requires exhaling–taking similar things and making them the same and deleting experiments that didn’t work out or at least moving them out of the core. Back in the olden days the limitations of the VM drove this, which is why the core concepts in Smalltalk were so refined. If you wanted space for your application you had to find a way to rationalize existing code.

On to Pharo. Pharo has a more coherent set of tools than the cacophony of Squeak but the base language/libraries are the same. The cleaner look was a relief although the paint splatters seem a bit indulgent. Whatever. I run the tests. They all break. Not a big surprise.

What’s broken? First, numbers. Syntax for literals. “16rff” in VW needs to be “16rFF” in Squeak. Sigh. No Double class because all floats are doubles, so Float is actually Double. No way to access the bytes in a Float. Then, dates. No built-in way to turn a Java timestamp into a timestamp. Different conversions required between Squeak and VW. Next, UTF-8 conversion. Grease to the rescue here. It’s the compatibility layer Seaside uses. Finally, sockets. At least this is no big deal. One class replaced by another. Except the Squeak version doesn’t support retrieving the position in a socket stream. Quickly fixed.

Now, I don’t particularly care for Java. Erich will tell you that I’m not productive until I’m swearing at Java. But credit where it’s due: the basics are portable. Numbers, sockets, strings, source code structures. You don’t have to worry about those moving from one platform or another. The vendors have found other battlefields on which to fight.

All the tests pass in Squeak finally. I’ve been making the changes in place, figuring I’ll make some kind of GratuitousDifferences class to capture the, well, you know. I load my code back into Monticello (I love SqueakSource, BTW–easy to use, free, what GitHub would have been if they’d stopped developing after a month. I told you not to blog when you’re mad.) I go to load into VW. Kaboom. The meta information is snarled. The monkey patches are gone. Merging does nothing sensible.

And that’s where I sit, blogging because I’m too mad to think. I’m not just mad because I’m stymied. That’s frustrating but not out of the ordinary. I’m a programmer after all. What I’m mad about is the gratuitous balkanization of Smalltalk. Here’s a language and platform that still has a lot to offer the programming world. There are lessons there about programmers being humans and users being humans that could be profitably absorbed by computing in general. And they won’t be, not if you can’t even move code between versions. Who would rationally invest in Smalltalk code if they weren’t already locked in?

I looked up Nash Equilibrium because it seemed appropriate. A Nash Equilibrium is (broadly speaking, look it up if you want accurate details) when none of the players in a game can change their strategy without losing something. The Mexican Standoff (as long as I’ve insulted Eastern Europeans I may as well insult our neighbors to the south as well) is a classic example. No one wants to be the first to lower their gun. The Smalltalk vendors seem to me to be in the same boat. Everyone has taken their “ANSI standard” Smalltalk in different incompatible directions, but to move towards compatibility would be a loss. Hence, no one is going to change.

The thing about a Nash Equilibrium is that what is rational from within the game can be absurd from an outside perspective. I’m calling bullshit on the state of Smalltalk. Vendors, you’re acting crazy. Have the tiniest possible core defined in terms of test cases. Build a shared library on top of that, implemented in terms of the core. Include numbers, collections, meta-objects, code structure, and code loading. None of this parcel/bundle/package/pundle/category nonsense. Compete on VMs, graphics libraries, and enterprise-y tools.

I know this can’t be a sensible proposal. There must be huge, insurmountable problems with it. After all, I’m too mad to think straight. But it would have made my job a lot easier today.

Written with all the love I can muster,

Kent

Categories: Open Source

Sonar 2.0 at Geneva JUG

Sonar - Wed, 02/10/2010 - 17:55

On the 23rd of February, Freddy is going to make a presentation at Geneva Java User Group.

Discussions will be of course about criteria to evaluate source code quality, but also about the evolution of the developer’s job. What does it mean today to be a Professional Developer ? What are the expectations and the challenges that a developer face on a daily basis ?

This event will be also a good opportunity to officially launch Sonar 2.0 and demonstrate the new features to cover the seventh quality axis : architecture & design.

For those who wish to attend (presentation is in French), you will need to register.

Thanks to the Geneva JUG organizers !

Categories: Open Source

Field Annotations Unexpectedly Rock

JUnit Max - Kent Beck - Fri, 02/05/2010 - 06:57

I’d like to introduce you to an unexpected effect Saff and I ran into the other night while working on JUnit: annotations on fields are much more powerful than annotations on classes or methods.

Suites

JUnit 4 introduced the use of annotations to tag methods that were intended to be executed as tests. We extended this idea to annotating classes to tell JUnit to run them in a non-standard manner. For example, the old JUnit test suite functionality now looks like this:

@RunWith(Suite.class)
@SuiteClasses({
  AssumptionTest.class,
  ...
})
public class AllTests {}

I always found this syntax ugly and awkward, but I didn’t have a better alternative that was declarative, easy to write, and easy to read.

Annotated Fields: Rules

A recent innovation in JUnit is the introduction of rules. We noticed that may people who implemented custom test runners (like the Suite runner above) we over-served by having to subclass one of JUnit’s existing runners. The extraneous detail made learning how to write runners harder than it needed to be and most requirements from custom runner could be met with just a few entry points. All customer runner writers need is a way to insert their code into the flow of control that results in a test running.

We created rules, annotated fields that are called before and after a test is run. Here is a rule that writes to the console when a test is about to run:

@Rule public MethodRule logger= new TestWatchman() {
  public void starting(FrameworkMethod method) {
    System.out.println(method.getName());
  };
};

We were surprised by how flexible rules turned out to be. People began writing new rules for all sorts of unanticipated purposes. We had stumbled on a really useful API.

Categories: Take 1

For many years people have been asking JUnit to support categories, being able to choose subsets of tests according to a user-generated classification scheme. I thought (and still do) that this is properly the job of the IDE, but since we had implemented everything else higher priority we gave categories a try.

@Category(Fast.class)
public class B {
  @Test
  public void c() {}
}
@RunWith(Categories.class)
@IncludeCategory(Fast.class)
@SuiteClasses({ A.class, B.class, C.class })
public class FastSuite {}

By tagging the class B as belonging to the Fast category, the FastSuite, tagged to include Fast tests, will run the tests in B but not A or C.

The API is ugly but functional. As we got into the twisty little corners of the implementation, though, I had a hard time following what was going on (the true measure of a design is whether the main flow remains clear after all the corner cases have been accounted for). I was ready to quit programming for the life of an itinerant goat cheese maker when I thought, “Maybe the problem isn’t the implementation, it’s the API!”

Categories: Take 2

Putting our experience with rules to use, Saff suggested we try annotating fields instead of classes. Categories are an example of the more general concept of a filter:

@RunWith(Suite.class)
public class FastSuite {
   @SuiteClasses
   public Classes classes= new Listed(A.class, B.class, C.class);
   @FilterWith
   Filter filter= CategoryFilter.include(Fast.class);
}

We transformed both class annotations into field annotations, one to tell the suite which classes to run and one to tell the suite how to filter out unwanted tests.

Here’s the magic part. As soon as we had written this we could imagine several other ways of filtering tests–by recency of failure, by name, and so on. Implementing these changes requires no modification of the Suite runner. In fact, these changes don’t require any modification of the core of JUnit at all.

Likewise, using an annotated field opened up possibilities for retrieving the classes to run as part of a suite. Johannes Link’s handy ClasspathSuite can be recast as an alternative implementation of Classes above, along with a host of other ways to find the classes to run.

Taking a Step Back

I’m a big fan of declarative expression in programming languages. Annotations are superior to convention (like naming all test methods “test…”) for declaring intent because they can be checked by the compiler. The addition of parameters to annotations makes them even more expressive. However, annotations attached to elements fully determined at compile time (like classes and methods) are fundamentally limited.

The problem is the coupling between intention and implementation. An annotation attached to a class specifies both intention (i.e. “run this class as a test in a certain way”) with an implementation (“this exact class right here”). Conflating intention and implementation limits the flexibility and power of annotations.

Annotations on fields separates intention and implementation. The intention is attached to the field while the implementation is specified by the value of the field, which can vary at runtime. It is this separation, and the flexibility it buys, that is the secret behind rules and this proposed new test suite semantics and syntax.

I’m still getting used to annotations and what they are capable of. I know I’ll be looking for ways to annotate fields in the future instead of classes or methods if I have the option.

Categories: Open Source

Sonar Gadgets for GateIn and Jira4

Sonar - Wed, 02/03/2010 - 15:10

Jérémi Joslin has recently published a series of open source gadgets for Sonar along with explanations on how he developed them. Those gadgets are aimed to work not only in Gatein but also in any OpenSocial/Gadget container. They have been tested in Jira4 and GateIn.

The development of those gadgets was incepted to follow Arnaud Heritier (the Software Factory Manager at eXo Plaform) vision on software development : “Developing a software with an increasing quality isn’t easy. Sonar allows us day after day, to analyze from a global dashboard up to the smallest line of code all quality indicators we extract from developments. But managing a product isn’t only being focused on its quality. It consists also to deliver on time, satisfy customers and community requests… To keep an eye on everything what could be better than a dashboard based on GateIn? With Sonar’s gadgets for GateIn we’ll be able to create our own set of dashboards to follow all metrics besides the activity of our teams, their impediments, tasks and issues on products (we are able, for example, to reuse Jira Gadgets in GateIn), and many more.”

Sonar Gadget in GateIn Dashboard:
Sonar Gadgets in GateIn Dashboard

Sonar Gadget in JIRA4:
Sonar Gadgets in Jira4 Dashboard
Links:

Categories: Open Source

The Responsive Design Project so far: January ‘10

JUnit Max - Kent Beck - Tue, 02/02/2010 - 22:06
In the Beginning

The second program I ever tried to write was a copy of the Star Trek simulator I had seen at the Lawrence Hall of Science. I had gotten through the BASIC language workbooks in a weekend. I had written a Roman numeral printer with no problems. How hard could Star Trek be?

I can remember my feelings sitting down at the teletype: anticipation, excitement, puzzlement, frustration, despair. I had no idea what to type. I knew the statements of the language, no problem. What I didn’t understand was design. I had no idea what elements there could be above the level of statements or how those elements could be related to produce the behavior I was imagining.

That day I just gave up and walked away. Over the next decade I slowly got a sense of what elements there could be in a program and how they could relate to each other. I acquired an aesthetic sense. I could take pleasure in a good design and feel discomfort in the presence of a poor one. Eventually I could take a problem and design software to address it.

But something was still missing. I couldn’t explain what I did when I designed. Little pieces like refactorings I learned, with a lot of collaboration, to articulate, but I couldn’t escape the nagging sense that design had hidden structure, like the plumbing and wires in the walls, structure that was critical to really understanding the process and product of design.

Revealing that hidden world of design is the goal of the Responsive Design Project. I want to understand and articulate the deep structure of software design. While the project has only just begun, I have some practical results I’d like to take you through briefly in this introductory article. And I’ll share with you the result that most disturbs and excites me:

  • Our illusion of control over software design is a dangerous conceit best abandoned.
In the Slightly Later Beginning

In 2005 I was invited to sit on a panel celebrating the 25th anniversary of the publication of Ed Yourdon and Larry Constantine’s Structured Design, the book that introduced the terms coupling and cohesion. I’d had a copy in my library since I’d used it as a college textbook, but, since I had used it as a college textbook, I hadn’t really read it. As I studied it in preparation for the panel, I realized that many of the questions I had about software design had been answered two decades before. This motivated me to make my informal study of design more structured and rigorous, and to communicate the results to a new generation of programmers. This was the official genesis of the Responsive Design Project.

Design is good. Design is central to effective software development. Programmers can add features steadily to well-designed software. Programmers can easily test well-designed software. Well-designed software is easy to tune for better performance. Most of the hard problems in programming turn out to be design problems.

But design has a dark side. While there isn’t a single best design for any system, there are many poor designs for that same system. The more experience a programmer has, the more design ideas he knows. The temptation is to put these design ideas in the system now because you just know you’ll need them eventually. Over-designing early leads to delaying feedback from real usage of the system, makes adding features more complicated, and makes adapting the design more difficult. By the same token, under-designing makes adding features more complicated, increases defects, and makes adapting the design more difficult.

The word “responsive” in the name of the project reflects this need for balance. Some design needs to be done in advance of coding, but over the life of the project most design will be done in response to the changing needs of the system and the changing and growing understanding of the developers.

The project is based on three parallel tracks of inquiry:

  • Quantitative. Measure real designs and see how they behave “by the numbers.”
  • Introspective. Carefully watch my own practice of design and draw lessons from the patterns I find.
  • Pedagogical. Teach software design and draw lessons from the students and their subsequent practice.
The Tour

Here is a quick overview of the lessons I have learned so far. Each topic deserves an essay of its own, or even a book chapter (say, you don’t know anyone who publishes technical books, do you?).

Beneficially Relating Elements. Designs are made up of elements that relate to each other in beneficial ways. Looked at actively, designing is creating and deleting elements, creating and deleting relationships, and increasing the benefit of existing relationships. Elements contains elements and so on down (and up).

Design is Fractal. There is no fundamental difference between implementation, design, and architecture. They are all a matter of beneficially relating elements. Learn to bounce between levels of abstraction. Most design takes place at a single level of abstraction but big advances often come as a result of rearranging the levels.

Safe Steps. Resolve the tension between efficiency and safety in favor of safety, then learn to take small, safe steps fast enough that from the outside it looks like you are flying.

Isolate Changes. Before making a change, isolate the area to be changed from the rest of the system so you can change an entire element at a time. For example, before changing a part of a procedure, extract the area to be changed into its own procedure. Make the change, then inline the changed sub-procedure if appropriate.

Embrace Ambiguity. Half done with a change and not sure what to do next? Stop and deploy (you can do this if you are working in safe steps). Complete the change when you know what to do.

Cultivate Confidence. Master your tools. Your feeling of mastery will improve your cognition.

Cultivate Humility. Try tools or techniques you aren’t comfortable with. Being aware of your limitations will improve your effectiveness.

Exploit Symmetries. Divide similar elements into identical parts and different parts.

Trust Succession. Design simply today, knowing you can add complexity needed tomorrow.

Design Is a Team Sport. The social side of design is as important as the technical side. Learn the skills needed to communicate and understand designs.

Play with Words. Recasting a design using a different metaphor can revolutionize your understanding.

Play with Pictures. Explain a design using hand-drawn pictures. Ask others to do the same. Design benefits from your whole mind: symbolic, verbal, visual, kinesthetic, conscious and unconscious.

Clear Strategies. All design changes use one of four strategies:

  • Leap. Make the new design and use it immediately.
  • Parallel. Make the new design and run both new and old simultaneously.
  • Stepping Stone. Create a platform to bring the desired new design within reach.
  • Simplification. Only implement part of the design now and add complexity bit-by-bit later.

Inside or Outside. Change the interface or the implementation but not both at the same time.

Toss It. Checkpoint often. If you get confused, learn what you can and revert. Now. Beware the sunk cost fallacy.

Keep It. If the system is working and creating value, keep it and improve it.

Start over. Sometimes the constraints of the problem plus the constraints of the existing system are just too much to handle at one time. I prefer incremental improvement, but sometimes it’s best to just wipe the slate clean and start over.

Both. Faced with design alternatives without a clear winner, do it every way. Just coding each alternative for an hour is more productive than arguing for days about which is better in theory, and a lot more satisfying.

Suit Design to Needs. A design strategy intended to maximize feedback from rapid changes looks very different from a design strategy intended to increase value by reducing cost. Both are appropriate at different phases of a system’s lifecycle.

Use Tension. There will always be tension between designing better and delivering sooner. Learn to fuel your creativity from this tension.

Conclusion

Many metrics in design are power-law distributed. For example, if you graph a histogram of the cyclomatic complexity of the methods in a large system on a log-log graph you get a straight line. In other words, there are many very simple methods and a few very complicated ones.

This same distribution is found in many measurements from nature—the intensity of avalanches, for example. This leads me to the most exciting and disturbing lesson I have learned about design so far:

  • Design is not a rational process.

Much as we like to think we are making design decisions, much of the structure of our systems emerges from the nature of the problem and the nature of the tools we use. Our designs shape us as much as we shape our designs. Insisting that we as designers are “in control” is counter-productive, leaving us wasting energy trying to act against nature. The challenge is to embrace responsibility and at the same time take an appropriate role in designing. Learning to achieve this balance is the next goal of the Responsive Design Project.

Categories: Open Source

Why NoSQL Now?

JUnit Max - Kent Beck - Sat, 01/30/2010 - 07:16

Why have NoSQL database reached the tipping point now? Almost twenty years ago I lived through the attack of the killer object databases. While they had some lovely technical superiorities (and I still make part of my living helping maintain a Gemstone/Smalltalk application), relational databases were able to beat object databases in the market. It wasn’t time.

Now, though, seemingly suddenly, alternative data models are all the rage. While my narcissistic programer self would love to believe this had something to do with technical superiority, experience argues otherwise. Instead, if you want to understand technical change it’s more effective to follow the money.

I was talking to a developer of a cloud-deployed application (hi Patrick, love that BrowserMob.com!) and our design discussions quickly focused on money. SimpleDB is great for high-transaction rate storage, but it’s too dang expensive for reporting. A special-purpose database just for reporting makes sense. It’s worth extra programming to reduce operational costs (the classic capex/opex tradeoff familiar to telecom engineers).

Some Made Up Numbers

Here are some made up numbers (thanks @jkordyback) to illustrate the dynamic (adjust the numbers for your actual situation). Say you have a database from a commercial vendor. It costs you annually:

$50K (license) + $1.5K (electricity) + $1K (capital) = $52.5K

If you need more performance it makes sense to get beefier hardware:

$50K + $2K + $3K = $55K

The performance advantages of an alternative data storage paradigm (column-oriented, document-oriented, key-value, map-reduce) don’t justify the additional cost and complexity.

Eliminate the license and the cost of electricity becomes a huge percentage of the cost of a database. (EC2 is basically a really complicated way of charging for electricity.) Any technical advantage that reduces energy usage turns directly into profit. Your database now costs you:

$0 + $1.5K + $1K = $2.5K

Buying beefier hardware is a giant bump:

$0 + $2K +$3K = $5K

What if you can avoid the hardware upgrade by shifting to an alternative database? Factor in internet-scale applications so you’re multiplying all your costs by 100 or 1000. The engineering required to shift to a different store or to keep multiply stores in sync vanish in comparison to the operational expense savings (1000 servers for illustration):

$0 + $1500K + $1000K = $2500K

Improving performance with hardware:

$0 + $2000K + $3000K = $5000K

Versus switching to a different store:

$0 + $1500K + $3000K + $500K (engineering cost) = $4000K

Implications

There are several things that catch my eye in this picture. One is that when I was going to school we were always taught, “In the olden days of computing, computers were expensive and programmers were cheap. Now it’s the reverse. Therefore…” We are back to the future. At internet scale, programmers are (sometimes) cheap compared to the cost of electricity. That’s a pretty fundamental assumption to change. I’m sure we haven’t fully digested the implications.

Another is that the technical advantages of alternative stores translate directly into economic advantage. If I was a big database vendor, I’d be diversifying away from reliance on big-iron licenses, say by buying a hardware company (oh…) or running a variety of storage models on my cloud (double oh…) Again, I’m sure we haven’t fully digested the implications of this shift.

In spite of the roughness of the numbers above, based on this I feel justified in my gut feel that the row-oriented relational model we’ve lived with for 30 years is about to shatter. Look for opex optimization to become an increasingly important topic for engineers. Look for vendors, both software and services, to deliver further opex improvements. Where it goes from there isn’t clear, but it certainly will be interesting. The time has come.

The “financial” “models” above are just thinking tools to look at trends. I’d love to see some real numbers and trends to validate the qualitative conclusions I’ve already jumped to.

Categories: Open Source

What does Open Source mean for SonarSource ?

Sonar - Wed, 01/27/2010 - 14:51

SonarSource, founded more than a year ago, is a Swiss company that leads the development of the Sonar platform. Obviously Sonar and SonarSource are really tight together : Sonar would not be where it is today without SonarSource, but the other way around is also true. Like any company making business around an Open Source product, we often get the question on what Open Source means for us and what is our real commitment towards it.

The short answer to this is a single word : LGPL. The is the license we chose from inception of the project instead of an ordinary GPL license. Why ? Because we believe that to make Sonar an extensible platform rather than just a tool, we need a license that fits both Open Source community and Commercial companies needs. To make sure people are going to invest in a platform, it should belong to its active users. With this choice and to keep its leadership on the platform, SonarSource has therefore committed to continuously invest in Sonar.

The longer answer refers to the idea of an Open Core by Jason Van Zyl. Jason describes what are his four principles and we fully adhere to them :

  • The Open Source product you provide to users must be great: the Open Core should stand on its own as something truly useful without any additional commercial add-ons. The software must perform well in a production environment.

    This is so true that many Sonar users don’t even know the existence of SonarSource

  • The Open Source product you provide should go through an ungodly amount of testing and QA. Testing and QA on the Open Core are the cornerstone of quality and should not be reserved for commercial versions of your product.

    The Sonar core is covered by about 1′300 unit tests and 150 integration tests (most of them are selenium tests) which are executed by two continuous integration server. Of course we run Sonar on Sonar on a daily basis and we do performance profiling before every release. SonarSource’s plugins are extensions of Sonar, not a kind of professional packaging : they fully depends on the quality of the core.

  • The Open Source product you provide should be architected such that all commercial features are plug-ins to the Open Core.

    The Views, Master project, PL/SQL and Identify plugin are fully based on Sonar extension points and nothing more.

  • The Open Source product you sell should have completely open pricing. If someone cannot clearly see what your pricing is and what the difference is between your open and commercial versions, you likely have a predatory and opportunistic pricing model

    I believe it is the case.



With the the adoption of LGPL and the respect of those four principles, you can definitely Come in, we’re open !.

Categories: Open Source

Write Your Own Fan Letter

JUnit Max - Kent Beck - Tue, 01/26/2010 - 08:00

During an exchange on the lean startups mailing list, one of the participants was having a hard time articulating the value of the vision he had for his proposed site. It was clear that he was passionate about what he wanted to build, that he was completely convinced about its value. The problem was that he couldn’t communicate the value to others.

I’ve been in a similar position many times, having a vision of where things were going next and how cool it could be but not able to communicate it clearly. I know three things about this situation 1) it’s incredibly frustrating for me, 2) it’s at least as frustrating for the people I’m trying to talk to, especially when I won’t shut up, and 3) I’m often wrong.

I’ve tried a variety of tactics to get out of being stuck. Just making the dang thing is one good way to clarify the situation. I wondered, though, if there was a less wasteful way for him to clarify his vision (I’m a geek and I don’t mind failing when I make something because making it is so much fun).

I borrowed an idea I’ve used often, speaking back from the future, and massaged it a bit: write a fan letter to yourself, written in the voice of a deliriously satisfied user. Here was my example:

Dear Match.com, Just writing to say how pleased I am with your service. I live in a small town and was unable to find dates. A friend told me about Match.com. The survey process required me to take a careful look at myself. Three days later I had made contact with 5 women in my area. After initial dates with two women, the third one was clearly a, well, match. We dated for six months and are now planning our wedding for this spring. The $40 I spent with you was the best investment I’ve ever made in my life. Yours truly, John Q. Geek

Visions start out in your head but they don’t matter until they touch someone else. Finding a path by which visions can create joy is one of the many challenges of entrepreneurship. The purpose of writing your own fan letter is to prepare your mind to bridge the gap between vision and effect. When you can imagine how one person’s life is changed by your vision, you are ready to move forward.

Categories: Open Source

Video Demonstration

IceScrum - Mon, 01/25/2010 - 16:21
Hi everyone, we are glad to show you the very first IceScrum² demonstration video! This video will guide you through the basic usage of IceScrum, from registration in the application to the creation of features, stories, releases and sprints. Please feel free to comment the video on this article or on the forum. Notes : Some features shown in the video (the sprint burnup, colored sprint backlog) are not in the R2#14.2 you can get in the download section of the website, but will be available in the next release of IceScrum. However, these features can be tried on the demonstration server. Discover the first video demonstration of IceScrum² Audio in french, english subtitles available.
Categories: Open Source

Not My Money Any More: Lessons from Poker 4

JUnit Max - Kent Beck - Fri, 01/15/2010 - 07:20

Here is my first lesson from poker. I take that back. My first lesson from poker is when two strangers ask if they can join the game, lose a little all night, then offer to cut the cards for $100 as the game is breaking up, count the deck before and then after they turn up the ace.

Okay, so this was my second lesson from poker: as soon as it leaves my hand it’s not my money any more. The day after the game I was incredibly frustrated. I’d lost and lost all night long. I kept rebuying and losing, rebuying and losing. All I could think about was when was the next game so I could win my money back.

Over the course of the next week I became obsessed by that thought: I was going to win my money back. At the same time, a part of me was standing off to one side say, “Excuse me, hello, this doesn’t make any sense.” At some point I figured out how to articulate what was wrong with my thinking–it wasn’t my money. It wasn’t my money as soon as someone else won the hand. In fact, it wasn’t my money as soon as I put my bet in. I gave my money to the game, the game gave the money to the winner, and that was that. I couldn’t win my money back because it wasn’t my money.

Peace. Sometimes obsession works like that. You figure out what’s wrong with your thinking and the cycle just stops. This was one of those times. It wasn’t my money. I couldn’t get it back, because it wasn’t mine. The next time I played, I was just playing. I started with $20 and went from there.

Many of you will have recognized the Sunk Cost Fallacy. Money already bet is gone, sunk. It makes no sense to factor it into your decision making. I knew about the Sunk Cost Fallacy, but I wasn’t spotting it in my own thinking.

The same situation comes up all the time in programming. You have a long refactoring session, every step of which is perfectly safe. Then you run the tests. A test breaks. What should you do? What should you do if the error isn’t obvious? What should you do if the error isn’t obvious and you have been refactoring for a minute? An hour? A day? Does the amount of time invested have anything to do with how you proceed?

My experience is that the right thing to do in a situation where I don’t know what I did to break the tests is immediately return to my last known green state and re-run the tests. Then I begin refactoring again and run the dang tests every step. But, and here’s the irrational part, the longer I’ve been refactoring, the more tempted I am to press on. That’s the Sunk Cost Fallacy at work.

The longer I’ve been refactoring, the less well I remember all the steps, the harder it will be to spot the error, the greater the value of just starting over. The harder it is to start over (emotionally), the more valuable it is to start over.

It’s not my time any more. Have I been refactoring for a minute? An hour? A day? Doesn’t matter. It’s not my time any more. I’ve given it to the program. It’s gone. What I should do now should not be influenced by the magnitude of time I already have invested. It’s not my time any more.

As I was writing this I occurred to me that I’ve been guilty of a variation on the Sunk Cost Fallacy with JUnit. If you’ve been reading this blog for any length of time, you’ve heard me whining about not having found a business model to turn the success of JUnit into revenue. The faulty reasoning goes like this: JUnit has created all of this value in the world (billions of dollars by my envelope calculations) and I haven’t received any of it.

So what? If JUnit hadn’t created any value at all so far but was still in the same position, what should I do? Exactly the same as if it had created a million dollars of value or a billion. It’s the Sunk Benefit Fallacy. What’s past is done. It’s not my benefit. All that matters is the current situation. Any energy or thought I expend on the past will only muddle my thinking.

So here’s my resolution. I will think about the current state of developer testing, figure out the trends, and get ahead of the curve. That’s how I will make a living from JUnit. Oh, and I’ll be sure to count the deck, or better yet, not gamble with strangers.

Categories: Open Source

2009 is over, what is coming up in 2010 for Sonar ?

Sonar - Wed, 01/13/2010 - 14:42

A change of year always gives to teams an opportunity to look back and measure what was accomplished… and then to start thinking of what the new year should be made of. I thought I’d share the output of the Sonar team retrospective.

At the end of 2008, very few people knew Sonar. The platform was made of a small community of early and eager adopters who were supporting the product strongly by giving feedback, asking for more functionality, making suggestions and testing new versions. It was also made of Sonar 1.5 that, looking back, was the foundation version of the platform. From this version, here is what was achieved in a year :

  • A dynamic development activity on Sonar core with 7 major releases since 1.5.
  • The transformation of Sonar from a tool to an extensible platform with more than 20 extension points.
  • More than 30 open source plugins have been build to extend Sonar core using those APIs, and more that are not open source.
  • the number monthly downloads has been multiplied by 10 during the year from 300 to 3,000.
  • Sonar has been given a heart called Squid that makes Sonar much more than an integration tool. Several metrics that do not exist elsewhere are calculated by Squid.
  • More than 4′000 emails exchanged on mailing lists and 1,000 Jira issues created.

So after all this, what could be an exciting challenge for 2010 ? We have set ourselves 2 very ambitious objectives for 2010 which should make the Sonar community continue growing :

  • Design analysis : we like to say that there are seven technical axes of code quality analysis (we call them the seven sins of the developer). Sonar currently covers sixth of them and the last one is for us the most important one with unit tests : Design & Architecture. Sonar 2.0 planned for February will start covering the 7th axis with O.O. metrics like LCOM4, RFC, DIT … cycles detection and DSM at package and class levels. All those information will be of course provided by Squid. Moreover, an architecture rule engine should quickly appear after Sonar 2.0.
  • Multi-languages : last but not least, give a real go at other languages. By the end of the year, we expect that plugins are available to cover properly : Java, PL/SQL, Flex, C/C++, Cobol, PHP and maybe more :-)

Here is a part of the program for 2010. I have now to leave you to start working on this as I think I will not have much spare time this year !

Categories: Open Source

The Pleasure of Pattern: Lessons from Poker 3

JUnit Max - Kent Beck - Mon, 01/11/2010 - 18:19

Studying poker has helped me crack a long-standing (maybe 20-year-old) puzzle for me: why are so many programmers musicians? I’ve sang and played guitar and various other stringed instruments since I was eight and I started programming at 13, so I’ve lived this dual life for a long time, but I’ve never understood it. I’ve heard explanations like “music is mathematical and programming is mathematical”, but I was never satisfied with them because they didn’t match my experience. I don’t think “mathlich” when I’m programming. Here are the three threads–programming, music, and poker–and how they came together for me.

Programming

One of the transformative experiences of my young life came in high school when Mr. Burishkin (hi Dave!) got mad at our algebra class (this was fairly common) and assigned us all 40 trig identities in the book. Usually we’d get half or a third of the exercises in a given set. Here was a big pile of work I couldn’t get out of easily.

For once I did the right thing and decided I would just plow through the whole stack. The first few were hard. I had to sweat to figure out how solve them. Each one posed unique challenges. At this rate the assignment was going to take me hours. Somewhere around half way through the assignment, though, a door opened in my mind. I’d look at a problem and see it in terms of the problems I’d already solved–I can split this in two, transform part 1 into this and part 2 into that and I’m done. I blew through the rest of the assignment in minutes, eager to solve more.

Not could I solve the problems quickly but I enjoyed the process. Every time I transformed the chaos of the original problem into a pattern I knew how to handle, it was intensely satisfying.

I have the same feeling of satisfaction with many programming-related tasks–when I see a big refactoring in terms of a succession of smaller refactorings, when I see a big task in terms of independently verifiable sub-tasks, when I see a big feature in terms of a succession of features each of which delivers value–at all these times I feel good.

Music

For thirty years I’ve played off and on with my musical partner Curtis Wright. We play American folk music. While I’ve enjoyed playing with many people, Curtis and I share a deeper level of musical communication than I’ve experienced with anyone else.

We don’t plan song endings. We end every song differently every time. Here’s how it works. When we’re really “on”, I will hear him play some interesting phrase somewhere in the middle of a song and something will click in my mind. There’s no explicit communication at that moment though–no nod, no eye contact, nothing. When we get to the end of the song, I will play that phrase as a tag and he will too, in perfect time. It is the most exhilarating feeling because we’re so completely at risk. If we chose different phrases to play, it would sound terrible and we’d look like fools. When we re-invent the ending and pull it off, though, it’s absolutely electric, both to play and to hear.

This is the most extreme example I have of matching patterns in music, but music is full of patterns–what makes a phrase, what makes a chord, what rhythm or rhythms are going on, what style should this piece be played in, backing up an unfamiliar tune by ear, and on and on. I enjoy music because I enjoy those moments of discovering the patterns in a piece. When I see a pattern, I feel satisfied. Woodshedding (practicing physical skills) is necessary to be able to take advantage of patterns when I see them, but I don’t enjoy it the way some people do. Experiencing music, for me, is experiencing patterns.

Poker

When I started studying poker I didn’t have any particular expectations, I was just interested. One of the basic problems in poker is having a more accurate belief about your opponent’s hand than they have about your hand. Their patterns of betting give you clues. At some point you say, “He must have just gotten a second queen,” because how he is playing doesn’t make sense otherwise. When I’ve guessed right, I noticed that I have a moment of the same kind of emotional pattern matching satisfaction that I have when I’m playing music or programming. That’s when I made the connection–this feeling is familiar, it happens when I program and when I play music.

There’s my answer: talent for music and programming occur together because accomplishment in each relies on enjoying seeing patterns. See a pattern, feel good, look for more patterns.

The bigger the stakes and the bigger the pattern, the greater the satisfaction. I won’t play poker for much money because I’m not any good at it. However, I experience more satisfaction when I see big patterns in programs (what some people call “architecture”) or big patterns in music (the so-called “long phrase”).

I suspect that my chaotic early life left me with a brain wired to crave moments of order and feel good when I find them. That and the innate ability to see patterns led me to activities where I got frequent mental rewards. I’ve just identified this connection, so I haven’t done any background studies into the neuro-science of pattern matching. From my experience I would guess that the same kind of dopamine release that accompanies other pleasurable activities happens in my brain when I match a pattern, musical or programmatical. I’d also like to see an fMRI to understand the neural activation patterns at such times and see if programming, music, and poker all really have similar physical neurostructures.

My partner Cynthia Andres is also a pattern matcher, but she matches patterns of human behavior. She has a good sense of what people are thinking or feeling based on cues that are totally invisible to me. I wonder if there are any studies correlating skill at reading people with skill at music. Are psychologists musicians the way programmers are musicians, or are musical and programming pattern matching similar in a way that musical and social pattern matching aren’t?

Another question this brings up for me is the evolutionary psychology of this kind of pattern matching. What is the evolutionary advantage of being able to match abstract patterns? Assuming it increases evolutionary fitness, what is the cost of it that kept it from becoming universal (I suppose my kids could provide a long list in answer to this question)?

Categories: Open Source

Variance: Lessons from Poker 2

JUnit Max - Kent Beck - Thu, 01/07/2010 - 10:19

One of the great things about styles of software development is that two people can hold absolutely contradictory opinions and they can both be right. Context is critical to what makes sense. (Hence the futility of searching for “best” practices.)

I was thinking about this today when thinking about our style of development for JUnit. I had tweeted that Saff and I were making “aesthetic” improvements. That’s how we work–we will add a few features, then spend a (seemingly) long time “cleaning up”, establishing symmetries, normalizing names, extracting and inlining components. At some point, we see the opportunity to add new features and progress is rapid.

Our style of development on JUnit delivers uneven results when viewed from the outside. We can go months without a new feature. Sometimes when we add features, though, it feels like we’re wearing jet packs.

One of the concepts in poker is variance, the magnitude of the swings suffered simply because of the effects of chance. Different strategies result in different amounts of variance. One strategy might result in small but steady winnings. Another would result in larger winnings but larger swings. The strategy with greater variance would only be appropriate for an emotionally stable person with a large bankroll.

Our JUnit strategy is high variance. High variance is not appropriate for all projects. If you’re making external commitments, you should work in a more predictable  way. Building and maintaining trust is more important than making occasional blazing progress.

We have the equivalent of a large bankroll because we make no external commitments beyond promising to continue improving JUnit. We enjoy improving the internals of JUnit as much as we enjoy releasing new features, so we aren’t bothered when there is a long gap between features.

How should you develop? As always, the answer begins with “it depends”. One of the factors it depends on is how much variance you are willing to support.

Categories: Open Source

Manipulation: A Matter of Intent

JUnit Max - Kent Beck - Tue, 01/05/2010 - 00:32

Since I was thinking recently about negotiation, I thought I would set down my thoughts on manipulation. This was in important issue for me to get clear because I grew up in such an intensely manipulative family. As an adult I caused a lot of damage being manipulative before figuring out there was a difference between manipulation and assertion. Keeping the distinction clear is important to effective negotiation.

Manipulation can result in a short term gain–you do what I want you to do. The problem with manipulation is the long-term effects. People aren’t stupid, not forever. If I manipulate you, you end up trusting me less. If we are in an important relationship, this lack of trust is costly. That’s the utilitarian argument against manipulation–it’s too expensive. An additional cost of manipulation is the personal cost–make a habit of manipulation and you come to see other people as objects instead of people. Since they stubbornly remain people, the gap between perception and reality is a significant overhead. Or you can rely on morality, that manipulation is simply wrong. This has the advantage that when you are tempted to manipulate, there is no short-/long-term tradeoff to tempt you.

I once asked a well-known figure in the software world, someone I respect and trust, about manipulation. He told me it was a matter of interpretation, much like the difference between and breeze and a draft. In both cases air is moving, the difference is up to the person experiencing the movement.

This definition is completely aresponsible (that is, lacking in responsibility). The problem with the definition is that it takes manipulation completely out of my control, but that doesn’t match my experience. If I talk about a topic, sometimes people take it as helpful and sometimes they take it as manipulation, but their reaction isn’t random. My thinking in discussing the topic influences their reaction, even though it doesn’t control their reaction.

Intent

What am I trying to achieve when I talk to you about, say, TDD? Am I trying to get you to try TDD so I can feel good about myself when you do what I advocate? Am I trying to get paid by your boss because I’m a consultant and I want to be seen as effective? Am I trying to positively influence my craft of software development so in twenty years I can look back with satisfaction on my career? These are all reasons I’ve advocated ideas, and they are all manipulative. What they all have in common is they are all self-centered. They are about me, not you.

Sometimes, though, when I’m talking about a programming topic, I’m just talking about that topic with others who really care. We each learn something. We might act differently as a result of our conversation, we might not, but no one will be offended either way. The conversation is a sharing of experience between people who care. The result is new knowledge, perspective, and deeper relationships.

Manipulation is intending for others to make less than a free and informed decision. If I have cars for sale (to pick another random example), I can sell them in a clear and direct way–help you understand your needs and resources and help you choose the best match between them. I can also sell in a manipulative way–try to get you to buy the car that makes me the most money. The flow of the conversation in each case might be similar. Many of the words and phrases might be identical. It’s the intent that’s difference.

Clues

How can you tell when you’re on the receiving end of manipulation? Intent becomes clear during conflict. Let’s say I’m telling you about TDD. You say, “All those tests would be too expensive to write.” How do I respond? If I’m being manipulative I’ll contradict you (“No they aren’t”), I’ll try to make you feel guilty or incompetent (“if you were a craftsperson you’d always right tests”), I’ll try to get you to ignore yourself (“C’mon, just try it.”) Would a friend do any of these things, someone you trust? No.

The question I have a harder time with is how to know when I’m being manipulative. At first I thought manipulation was just how everyone acted, and I interpreted everyone else’s behavior on that basis even when they weren’t being manipulative. Eventually, with a lot of helpful (i.e. painful) feedback, I started to learn that I could trust other people to make their own decisions and I would still be safe.

I still have the habit of a lifetime. When conflict becomes apparent, I know that I tend towards manipulation so I watch my own reactions. Do I feel angry? Frustrated? Fearful? The early signs of these emotions can be subtle–a twitching muscle, tightness around my mouth, my patterns of eye contact. These emotions tell me that I’m not trusting the other party to make their own decision. Unless I consciously choose otherwise, my arsenal of manipulative techniques will come next.

When I’m aware of myself and I encounter conflict, I remind myself that I can trust others to make their own decisions. I make a point of asking questions to understand their point of view. Where does it come from? What experiences led to it? What motivations lie behind it?

Maybe this sounds like a soft technique. Sometimes it is, but sometimes it isn’t. I can choose to be clear and direct even when in the presence of someone I don’t understand at all. I can try to understand without agreeing with them. Fortunately, I’m seldom in the presence of stark, raving lunacy. More often (maybe I should say “always”) there is a difference of opinion, as in the case of the cost of tests. Both parties have reasons for what they think. Maybe there is still a worthwhile exchange, maybe there isn’t, but I can always choose to help the other person make a free and informed decision.

I’ll come back to this again and again: we are, all of us, surrounded with unidentified, valuable possibilities every day. If I maintain the intent of helping others make free and informed decisions and maintain my own intent to make free and informed decisions, much of the time there is the possibility of a valuable exchange. If this is one of the times when there isn’t, okay. Move on. In any case I will have learned something by really listening and I will have given the other party the opportunity to learn something from me. There really isn’t any way to lose, unless I panic and pick up the puppet strings, strings that aren’t really attached to anything.

Categories: Open Source

Information is Money: Lessons from Poker 1

JUnit Max - Kent Beck - Mon, 01/04/2010 - 07:47

I’ve been unknowingly following good advice for keeping my brain fresh into middle age: studying topics that challenge my thinking. Podcasts during chores are a great time to study. The difficult concepts I listen to shake me up and keep my brain moving.

My latest obsession (my son’s word) is poker. I started playing with friends a couple of years ago, but I recently began actually studying. I’m still a beginner but I’ve noticed that my business thinking is changing based on what I’ve learned. I’ll try to relate the changes as I experience them.

Here’s the first. I was talking with a new client about how we could get started with our relationship. We were looking for something small to do first and we settled on having me present a webinar on Responsive Design. He asked how much it would cost for a one-hour presentation with followup email for questions. I quoted him a price. His response, “Oh, that’s cheap!”

Here’s where poker thinking helped. Before studying poker I would have started kicking myself–why oh why didn’t I ask for more? Simply put, I didn’t know what the market was for webinars. I couldn’t have known to ask for more, but that wouldn’t have stopped me from self-recrimination.

Rather than be frustrated, though, with my poker hat on, I was satisfied with the transaction. No need to kick myself. I had made a bet by offering my time. In return I got clear information–I could charge much more. The information I received was worth far more than the money I “lost” by not asking for more. I was out two hours (at a “reduced” rate), but I was up the future increase in earnings.

Studying poker has helped me establish the monetary value of information much more clearly than I ever have before. In poker, it’s common to trade money for information. By getting used to such exchanges, I am more relaxed in business.

Oh, and if you’d like a webinar, I’m available. My bid will be more than it would have been a month ago, but we can talk.

Categories: Open Source