Skip to content

The Evil Guide to Getting Papers in Empirical Software Engineering

01/02/2012
Over the past years as a PhD student I had many discussions about how to best publish papers. After many rants about the in my opinion deprecated review system (not really my point in this post, but it is always good to plant and nurture the seed ;)), I came up with the following evil paper publishing scheme. For all of you that are still innocent or are easily tainted by evil schemes and thought, you might want to stop reading right here.

Here is what you should do:

  1. Take a course in creative writing.
  2. Read paper from researchers that do studies in industrial settings.
  3. Read paper the open source/open data counter part to the papers describing studies in industrial settings.
  4. Read open source/open data papers that do not have an industrial settings counter part.
  5. Write the the industrial settings counter part and publish.

The Trick that makes it all work:
Have you ever read a paper that reported on a study conducted at a company that wishes to remain anonymous? Right, that is the loophole that we are exploiting by simply stating that we cannot reveal our sources and therefore take any means of checking our research. Worst case we can always refer that our results differ that much because of the very special circumstances of the project we studies.

Step 1: Creative Writing
Create writing is the one skill you will need the most to succeed with this evil scheme. Creative writing will help you to come up with nice ideas and construct an intuitively logical argument (pepper it with some sound stats on face data and nobody will be able to stop you). The next steps will help you to find topics to apply your creative writing and make sure that you stay within your limits and don’t start to make outrageous claims in your papers.

Step 2: Industrial Settings
This you will need to do to see how data from industrial studies is reported and to get a good overview about the arguments used why they cannot name the organization or publish the original data. Furthermore, industrial case studies often have some very unique settings. Those settings are always a good source of inspiration when trying to justify some of your variation in results and give you a bit more credibility that you actually did research at a company.

Step 3: Open Source/Data Settings
To further your understanding between studies done at industrial sites that don’t allow for publishing their data and the rest of the world (well that is mainly the open source community)you should read up on studies whose data is available. This will give you a better idea on what data usually is available to the public and what can be done with it, as well as short comings that you might not find in industrial settings. For instance, in open source you have a more difficult time getting access to developer in particular when you want to shadow them during their development time, whereas in an industrial project you usually know where the developers are located and when they are working.

Step 4: No Industrial Settings
Next on the list is to get ideas for you own “paper”. The easiest way is to read papers about studies that do not have a closed data equivalent. You basically scour the huge amount of studies that had certain short comings due to their data collection methods or did not explore the connection between the project setting and the results in great detail and use this and other potential differences to write your own paper.

Step 5: Write Your Paper
When writing you paper that is based of the ideas of studies that are already published it is important to vary the results and the setting in a meaningful way, such that your results can intuitively be explained by the variation in the setting. This serves two purposes, you won’t make any claims that are unbelievable and stay within a certain norm of accepted results and you will make the researchers of the original study happy as to mostly confirming their findings.

Bonus: Steps 2-5 are not essential
Now for the creative writing pro’s that have enough experience in their field of study. You can omit steps 2 to 5 and simply conjure up a study that is generally believable and follows the trends within your community. The trick is to stay away from things that might create doubt in the validity of your study by making outrageous claims or contradicting accepted norms, simply put: keep a low profile. This won’t win you any best paper awards but a paper is a paper.

The final questions for you: How important do you think is it to teach graduate students and researchers ethical guidelines/behaviour? And how do you and/or institution do that?

8 Comments leave one →
  1. 01/02/2012 14:43

    Well now you’ve spoiled it for everyone, Mr. Blabbermouth.

  2. 02/02/2012 16:10

    A corollary of this should be “better to do lots of mediocre things than one thing that’s really good. That way, people will know you were trying, even if you are a little dumb.”

  3. 03/02/2012 06:07

    Wow Adriano..magnifico..

    • 03/02/2012 08:20

      thanks, Indi

      • 07/02/2012 09:15

        Btw.. we just had a discussion with profs today about the easiest way to get a paper accepted.. Copy paste an existing work, introduce some spelling errors in the abstract and submit it to B venues..har har har.. Somebody in this green earth actually got away with it..

      • 07/02/2012 11:11

        that is a good one too!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: