Sunday, August 10, 2014

The "must be a new guy" communication issue

All teams, whether bigger or smaller, suffer from communication issues. But when teams are organizations as huge as states, then funny things start to happen. This is such an anecdote.

A few days ago I had to legalize some spanish tax documents in order to present them to the greek tax authorities. Legalize a document actually means get some official sign  and ink-stamp it. On top of that you need the apostille of the Hague, which essentially is another paper confirming that the original paper is indeed signed from the authorized person.

At that point I'd like to note that the documents where already electronically verifiable by a unique hash. We live in an era with so much talking about secure authentication, verification etc when there are still whole countries out there that base their authentication on ridiculously easy to forge mechanisms. And on a convention from 1961. But that's another issue.

In any case, I had to go first to the Ministry of Finance to get the documents signed and the to the Ministry of Justice to get the apostille of  the Hague, or should I say the plague. So I'm at the Ministry of Finance explaining to the two nice old ladies what I need and they tell me to wait until they get my papers ready. They come back to me after a few minutes with the signed papers and I was ready to go. But before I leave, I notice that although the document was like four or five pages long only the first one was signed.

This astute observation was the result of some hard earned skills I developed while coping with the public sector in Greece. Everything boils down to one rule: never trust the public servants and double check your documents.

Of course, I go back and ask them if that's ok and if there will be a problem for having the apostille on the whole document. They reassure me that this is the normal procedure and that there'll be no problem with that. My guts don't believe a thing but what can I do. After all, I'm in a different country now, maybe things here work, who knows.

After waiting a super long queue at the Ministry of Justice, my turn comes and I find my self sitting in front of one of those desks with a bored autistically bureaucratic clerk behind them. I explain to him what I want and he tells me that it can't be done (of course) and he can only verify the first page of the document. "But they told me this is how it's done...", I mumbled, in a desperate attempt to save myself of repeating this process only to receive the counterargument: "You know it's vacation period, there are a lot of substitutes. The person who told you that, must be new.". "Not on this earth" I thought, recalling the two old ladies.

Well, there I was again on the Ministry of Finance, with the one lady that was still in the office, trying to explain to her that the document should be signed in every page. Fortunately it didn't take too much convincing her, all she had to do is sign a couple more pages. Nevertheless she seemed troubled with the behavior of the other clerk and she stunned me with her remark: "You know it's vacation period, there are a lot of substitutes. The person who told you that, must be new.". At least there is a consistency on that.

Labels: ,

Saturday, June 21, 2014

Coupling and Cohesion

In software engineering we've been taught that coupling should be low and cohesion high. But there is a lot more depth and detail in these two seemingly simple measures of software quality. There are a lot of very informative sources about them around the net. This is a gleaning of some ideas and examples that hopefully makes sense.

Coupling

Definition: The degree to which each program module relies on each one of the other modules.

Layers of coupling
Layers of coupling. (source wikipedia, license cc by-sa 3.0)
Layers of coupling (from high to low)
  • Content: One component uses contents of another 
  • Common:  Share global data
  • External:  Share imposed data format, protocol or device interface
  • Control: Components control the flow of another, by passing it information on what to do
  • Stamp: Share a composite data structure and use only a part of it
  • Data: Share data through, for example, parameters
  • Message
  • Uncoupled

Content Coupling
  • Component directly modifies another’s data
  • e.g. Part of program handles lookup for customer. When customer not found, component adds customer by directly modifying the contents of the data structure containing customer data.
  • Improvement: When customer not found, component calls the AddCustomer() method that is responsible for maintaining customer data.
Common Coupling
  • Two or more modules have read and write access to the same global data
  • Usually a poor design choice because:
    • Lack of clear responsibility for the data
    • Reduces readability
    • Difficult to determine all the components that affect a data element (reduces maintainability)
    • Difficult to reuse components
    • Reduces ability to control data accesses
  • It’s ok if just one module is writing the global data and all other modules have read-only access to it.
  • e.g. Process control component maintains current data about state of operation. Gets data from multiple sources. Supplies data to multiple sinks.
  • Improvement: Data manager component is responsible for data in data store. Processes send data to and request data from data manager.
Control Coupling
  • One module tells another what to do with a control flag.
  • Bad when component must be aware of internal structure and logic of another module (can't be reused)
  • Good if parameters allow factoring and reuse of functionality
  • Acceptable: Module p calls module q and q passes back flag that says it cannot complete the task, then q is passing data
  • Not Acceptable: Module p calls module q and q passes back flag that says it cannot complete the task and, as a result, writes a specific message.
Stamp Coupling:
  • Two modules are stamp coupled if they communicate via a passed data structure that contains more information than necessary for them to perform their functions.
  • Requires second component to know how to manipulate the data structure (e.g., needs to know about implementation)
  • May be necessary due to efficiency factors: this is a choice made by insightful designer, not lazy programmer. 
  • e.g.  Customer Billing System. The print routine of the customer billing accepts a customer data structure as an argument, parses it, and prints the name, address, and billing information.
  • Improvement: The print routine takes the customer name, address, and billing information as an argument.
Data Coupling
  • Modules communicate by passing simple arguments or data structure in which all elements are used
  • Good, if it can be achieved. 
  • Easy to write contracts for this and modify component independently. 

Cohesion

Definition: The degree to which all elements of a component are directed towards a single task and all elements directed towards that task are contained in a single component.
 
Layers of cohesion (from low to high)
  • Coincidental: Arbitrary grouping of elements
  • Logical: Elements are logically categorized to do the same thing
  • Temporal:  Elements of a component are related by execution timing
  • Procedural:  Elements of a component are related only to ensure a particular order of execution
  • Communicational: Parts of a module are grouped because they operate on the same data
  • Sequential:  Output data from one element serves as input data to the next
  • Functional: Parts of a module are grouped because they all contribute to a single well-defined task

Coincidental Cohesion:
  • A module has coincidental cohesion if its elements have no meaningful relationship to one another.
  • Elements needed to achieve some functionality are scattered throughout the system.
  • Worst form. It's bad m'kay?
  • "However, one interesting property of coincidental cohesion is that even though the code in question should not be stuck together, it tends to remain in that state because programmers are too afraid to touch it." [source]
  • e.g. Utilities
Logical Cohesion: [link]
  • A logically cohesive module is one whose elements perform similar activities and in which the activities to be executed are chosen from outside the module (= from the client).
  • e.g.  A component reads inputs from tape, disk, and network. All the code for these functions are in the same component. Operations are related, but the functions are significantly different.
  • Improvement: A device component has a read operation that is overridden by sub-class components. The tape sub-class reads from tape. The disk sub-class reads from disk. The network sub-class reads from the network. 
Temporal Cohesion:
  • A temporally cohesive module is one whose elements are functions that are related in time.
  • Difficult to change because you may have to look at numerous components when a change in a data structure is made.
  • Component unlikely to be reusable.
  • A module called "On_Really_Bad_Failure" that is invoked when a Really_Bad_Failure happens. The module performs several tasks that are not functionally similar or logically related, but all tasks need to happen at the moment when the failure occurs.
  • Improvement: Put the components of the code on where they functionally belong and use a routine to invoke each component on failure.
Procedural Cohesion: [link]
  • A procedurally cohesive module is one whose elements are involved in different activities, but the activities are sequential. 
  • Procedural cohesion is essentially temporal cohesion with the added restriction that all the parts of the module correspond to a related action sequence in the program
  • Actions are still weakly connected and unlikely to be reusable
  • Related to bad control coupling?
  • e.g. Same module reads customer entry from db & sends email if he's overdue.
  • Improvement: Let db module handle reading and mailer module send the email.
Communicational Cohesion: [link]
  • Communicational cohesion occurs when a module performs operations related to a sequence of steps performed in the program (see procedural cohesion) AND all the actions performed by the module are performed on the same data
  • Communicational cohesion is an improvement on procedural cohesion because all the operations are performed on the same data
  • Communicational cohesion isn't common in object-oriented systems since object-oriented designs tend to stress polymorphism.
  • e.g. Stack module: push(), pop(), empty() etc.
  • e.g. Book module: obtain author, title, or price of book from bibliographic record
Sequentially Cohesion:
  • A sequentially cohesive module is one whose functions are relaed such that output data from one function serves as input data to the next function like an assembly line.
  • Occurs naturally in functional programming languages.
  • e.g. Retrieve customer, retrieve customer order, and generate invoice
Functional Cohesion: 
  • A functionally cohesive module is one in which all of the elements contribute to a single, well-defined task. Object-oriented languages tend to support this level of cohesion better than earlier languages do.
  • Reusability:  Modules such as read_file, or draw_graph are more likely to be applicable to another project than one called initialize_data.
  • Fault isolation: If the data is not being read from the file correctly, there is a good chance the error lies in the read_file module/function
  • e.g.  Calculate Net Pay module: calculate taxable deductions, calculate tax, calculate social security deduction

Rule of "one sentence module description" [source]

One way to determine the level of module cohesion is to write a concise sentence that describes the module's functionality. You can usually tell from the sentence structure the level of cohesion. 
  • "Calculate Invoice Total" => Functional
  • "Perform an activity depending on the menu option" => Logical (switch)
  • "Compute minimum, maximum, and average rainfall" => Communicational
  • "Edit all input data" => Procedural
  • "Retrieve and (then) validate employee information" => Sequential
  • "Process various functions depending on the value of the parameters passed" => Coincidental
  • "Validate Customer Details" => Functional
  • "Perform end of day activities" => Temporal
 

Sources: [1][2][3][4][5][6][7]

Labels: , , , ,

Tuesday, December 17, 2013

Lean Startup Machine Madrid

The weekend of 18-20 of October the Lean Startup Machine held a workshop here in Madrid. I had booked the ticket as a super early bird back in July, thanks to Pierre from guiripreneurs. Last week I was thinking of selling my ticket but then I thought what the heck, give it a try and get out of your comfort zone.

I had no preconception on what to expect neither did I went prepared enough. I am working for a start-up  and I'd heard before of Steve Blank and Eric Ries but other than that, tabula rasa.

Overall I found the experience totally worthy. A lot of interesting talks throughout the weekend and most importantly experience gained. The only thing that I found a bit frustrating it was that the workshop was in spanish and not in english.

Friday Night Pitch

On Friday night you get to make the pitch in fifty seconds which is a very short time when you are almost trembling. Nevertheless I decided to pitch. For me it was like riding a bicycle. If you lose your momentum you fall down. I barely managed through.

The idea I pitched for was simple: use energy consumption data in order to deduct personalized energy-saving recommendations for households. It's actually not such an original idea that came from discussions with a friend of mine that has done a lot of cool stuff about smart energy meters. What really got me is that the, yet unproven, software solutions are based on machine learning. Here is a cool looking one. Since I come from a scientific background I am naturally interested when someone tries to mix state-of-the-art science and business.

There were like thirty pitches and then the fifteen most popular ideas were voted. Mine was among the ones that made it, ranking first, to my great surprise.  So as the "team leader" I had to choose another five to work with me. More people were interested so I had to let some down.

Meet the team

At the moment I thought we were actually expected to build an MVP so I promptly opted for two young tech guys (Xavi and Jon) and a designer (Andres). . The other two members of the team were a project manager in an established company (Isabel) and an entrepreneur (Gustavo). It seemed just the right mix and after this is over I can say that everyone was surprisingly talented. One thing we were missing though, we had no idea what we were supposed to do. For me it was more like a hackathon. It isn't obviously. Each team member seemed to have their own preconceptions and were, likewise, kind of misled. We found out later it that the Lean Start-up Machine process has to do more with finding if your idea resonates with the customer base and not so much about it's execution or other things.

Our team. From left to right: Gustavo, Andres, me, Isabel, Jon and Xavi.

 

Vadlidation Board

After the teams were formed we were given a board with which we were supposed to work with. It is named Validation Board and supposedly helps team manage their ideas, take action and make informed decisions based on metrics. That's a nice Idea but a very platonic one since we weren't given any sufficient explanation on what the boxes really meant or with what we should fill them.

But let me explain how - I think - this board works. First of all you have to make two hypotheses. The first is about the customer segment you want to focus on. The second hypothesis is to identify a problem that they have, stemming probably from your own experience. After that you have to make a series of assumptions about that customer-problem pair. From all those assumption you choose the riskiest one and set a validation criterion. Then you go out of the building and interact with people in order to see if the criterion holds true in the real world. If your assumption is validated then you propose a solution hypothesis again making assumptions about it and validating the riskiest one by asking more people. If an assumption is invalidated then you go back and change what's wrong in your hypotheses. The whole assumption process is then repeated until you come up with a customer-problem-solution trio that gets validated. Then you can start developing the solution.

Our validation board.
Yes I know it sounds like too generic and arbitrary. From my teaching experience, I came to find out that when a student cannot understand something, two things might happen. Either the teacher doesn't really know their material or that the material is far too advanced for the student. In this case I believe there is no material at all I'm afraid. This became more apparent when we asked some of the mentors to give us some specific guidance and all we got was some one-size-fits-all responses.

By no way this means that there is no value in the Lean Startup Machine though, all the contrary. It just means that it's not something that can be taught in the academic sense of the term. It's a more hands on approach that you get to digest with practice. The validation board helps give a bit of structure to this continuous process, making it easier to identify where you go wrong. But it can't protect you from uncertainty, wrong assumptions, your biases and all the other real world obstacles. Unfortunately, apart from one or maybe two speeches, I didn't see a lot on advice on good practices that help tackle this obstacles.

Guidelines to get the most out of personal interviews with customers. From Jose Antonio de Miguel of yoemprendo.es

TL;DR

If I were to sum up the lessons learned this weekend it will be something like this:
  1. Trust the data: Don't trust your own - biased - hypothesis of what people may want, ask them first.
  2. Find worthy data: It's a continuous process. Repetition, practice, keeping and open mind and repetition makes better in extracting useful information and decreasing uncertainty.
  3. Validate first: Don't spend a minute on developing a product before you are sure people want it.


Hard working teams.

Labels: , ,

Friday, September 6, 2013

I am a published scientist.

This post is to celebrate the publication of my first scientific paper. After a year of hard work it's out there, on the loose. A tiny contribution to the vast corpus of human knowledge. I must admit it's really satisfying. Minus one on the bucket list.

In a few words, the goal was to build a real-time system for human detection in order to be used on board a medium sized mobile robot, the Summit XL(c) by Robotnik(c).
Summit XL
Summit XL(c)
The task was accomplished by using two sensors: a Hokuyo UTM-30LX-EW(c) and a PointGrey Firefly MV(c) camera. The information from the two sensors was synchronously fused in order to produce greater accuracy. The technical details can be seen on the paper which was published on the  Sensors journal. It has an open access policy which is very important for the dissemination of knowledge, but that's another discussion.

The system was initially prototyped with Matlab(c) and Python. The real-time version was later developed in C++ using the Robot Operating System and OpenCV libraries. The source code has been released on github and there are plans to create an official ROS package of our group in UPM. Speaking of which I am really happy to have worked with colleagues such as Mario, Antonio, Andrés and Pablo here.

Some of the results can be seen in the following video:


Labels: , , , , , ,

Tuesday, October 9, 2012

Matlab API for Hokuyo UTM-30LX-EW

We recently acquired a new Hokuyo UTM-30LX-EW laser range finder. It's based on the old UTM-30LX laser so that means it has an effective range of 30m and 270 degrees. The only differences with the previous model are:

  • Ethernet connection instead of USB
  • Multi-echo functionality, very helpful for outdoors
Anyway since it's a fairly new laser, I couldn't find any drivers or API to use with matlab. So I decided to write my own. It's very simple and the only thing it does is initialize a connection to the laser and then starts grabbing the data. It also incorporates decoding and error detection functions. The code runs at almost 40hz. You can find it uploaded at mathworks file exchange.

UPDATE:
My friend Ana form KIT has upgraded the API with the capability to also return intensities and multi-echo data. You can find it here.

Labels: ,

Thursday, July 26, 2012

Laser-camera calibration

Hokuyo URG-04-LX & Firefly-MV
In my project I use a simple firefly-mv camera and a laser range finder from Hokuyo. The laser that I use now is UTM-30-LX-EW instead of the previously used URG-04-LX. More details here.These two sensors are placed on a moving robot and they are used in conjuction to detect pedestrians (and other moving objects  in the future). Since we use data from both sensors to extract information about the presence of a pedestrin, we need an accurate way of correlating, both in space and time, those two signals. Solving the time issue is trivial with the use of timestamps which give adequate accuracy for objects moving at relatively low speeds.
But for the space correlation things are more complex. The camera singal is a projection of a 3d plane into 2d and the laser signal is a 1d scan from the 3d world. So if you want to know which points are corresponding you have to know the exact 'difference' between the point of view of the two sensors. That is we need to know the exact transformation matrix between the laser and the camera. This process is called laser-camera calibration. And to do it we are going to use a neat matlab toolbox developed by A. Kassir[3].
Figure1.
Detected corners on the checkerboard
First we begin with intrisincally calibrating the camera[1]. The optics and electronics of the camera are far from perfect, so some distortion is apparent. What we need to find is the distortion coefficients so we can later "repair" each camera image. To do this we use a checkerboard pattern with known dimensions. We use the checkerboard because it's easy to detect the corners and thus compute the plane in which they belong (see Fig 1). For more accurate results we have to move the checkerboard into various positions and orientations (Fig 2). 
Figure2.
The algorithm extracted the various positions of the checkerboard
After the camera is intrinsically calibrated, next comes the extrinsic laser-camera calibration which will give the distance and rotation between the two sensors. For this we have already recorder laser scans for each camera photo grabbed. The laser segment that corresponds to the checkerboard is automatically extracted [3] and the problem left to solve is how to minimize the algebraic distance of the two measurements. Which is a linear problem and can easily be solved with least squares. A non-linear method based on the plane orientation is also used, along with a global optimization step including all the poses of the checkeboard [2]. After we get our results we can superimpose laser scans on images.
Laser segments superimposed on the camera image
The calibration is not so easy two be done and the results sometimes are subpar what is really necessary. One detail I can't stress more is the need for a really firm base to keep the camera and the laser from moving in relation to each other. The good news is that if you have a 3d printer you can easily design and print your own base. We (Mario that is) designed two bases one for the urg-04-lx and one for the utm-30-lx-ew, the camera is firefly-mv for both. You can download them here in ipt (autodesk inventor) and stl format, in case anyone finds them useful (yes I'm refering to you my future self).


[1]. J.-Y. Bouguet. Camera Calibration Toolbox for Matlab, 2009 

Labels: , , , ,

Sunday, June 24, 2012

My first steps: Day one

The "package"

This is the package that came from xyzprinters.com. It's so nice and compact. It contains almost everything we'll need to build my printer. Below you can see the plastic parts that Juan gave me. They are of superb quality.
The box :) 
Showing all the stuff
The printed parts

Here comes the first trick

So this is one of the most important things you have to know about printable plastics. You see these plastic cases, they were broken when we put the bearings inside them. Fortunately it's easy to fix this because the plastic is soluble to aceton.

Broken plastic? Don't worry use aceton.
Aceton is a chemical liquid very easy to find. Your girlfriend uses it to clean her nails from polish. But since you are here it seems very unlikely to have a girlfriend, and if you do she probably doesn't have nails and/or hands. This is no time for worries, though, for you can buy it in every supermarket or hardware store. I mean aceton not an able-bodied girlfriend.

Our good friend aceton
You can use a small brush to apply it where you need. The plastic will glue by itself in a few moments, but you'd better give it some time to properly dry.

Good as new
Just give it some time to dry
Finally the bearing inside their cases

The prusa triangles

For the most part of the chassis you can use Gary Hodgson's visual instructions. I use Juan's verbal instructions because there are a few things that have changed in prusa mendel iteration 2 from the original model.
So the best way to begin with is the two main triangles. Be careful with how you place the washers and the nuts cause if you do a mistake the later it sucks to dismantle the whole thing. A nice trick for aligning the two triangles is to use the rest of the rods as guides. Just for reference the nuts are size 13. 

Bottom plastic and threader rods
Use some other rods to help you with the alignment
The two main triangles
Don't tight the nuts too much, you'll need to make adjustments later
Again use some rods to make sure the alignment is ok

Front and back

This is the back and front of the printer. The motor of the y axis goes to the front. Be very careful with the washers and stuf. There is a difference here from iteration one. That extra nut between the big black plastic and the bearing is no longer needed. This also makes the aligning process later, easier.

The motor of the y-axis will be screwed to the black plastic
Another view of the front part
The triangles with the front part added
The back part also added
The upper part is finally added. It will hold the z-axis parts

End of day one

Happy working class people