printf("Adieu Ritchie, McCarthy and Jobs");

In the last two weeks, the world of computing has lost three of its stalwarts - Dennis Ritchie (the inventor of C and Unix), John McCarthy (the father of AI and Lisp) and Steve Jobs (of Apple). 

For me and for basically everybody I know of, the introduction to programming came through C. I still admire C for its simplicity and orthogonality. The notions of elegant and efficient programming can probably not be taught better in any language other than C. Unix, of course, is a boon to the world - an operating system so simple and elegant that every OS of note essentials follows its design principles. Ritchie, Brian Kerninghan and Ken Thompson were first class hackers who changed the world  forever. 

The very term 'Artificial Intelligence' was coined by John McCarthy and he along with Marvin Minsky, Claude Shannon laid the foundations of this branch of computer science that I now happen to be associated with. AI is now the darling of the computing world, though the challenges McCarthy had set for us is still far away. And who can forget Lisp - the language that has spawned the functional paradigm of programming that every programmer of note will appreciate for its elegant style.     

Unlike Ritchie and McCarthy, Jobs was no technical whiz or geek - but the idea that drove him was the same - beauty and elegance. Its a word not normally associated with CEO - but it is his insistence of aesthetics that has seen Apple build the sleekest products that just do what they are supposed to do with no feature crap that is the bane of all software. 

R.I.P

Python is great!

Recently, I learnt Python mainly in the course of doing some web programming, not wanting to jump into the jargon laden Java technology stack. Also, I wanted to learn a language for quickly coding out things, yet be able to manage and organize code as well as in Java. Quite a lot of people have taken to Python for these reasons, and it surely doesn't disappoint. Coding feels like a breeze, yet I can write well organized and structured code. The language is powerful and expressive and the list manipulation facilities are a breeze. There is a hint of functional programming too with 'map', 'reduce', 'filter' functions. It has a good collection of libraries too, and almost every functionality has Python APIs. So there is little reason to be stuck with Java because of non-availability of libraries. The only odd thing is that the language has no braces, and relies on indentation for defining scopes. Enjoying Python for now, hope to use it heavily in times to come !

The disposable Ph.D

The Economist recently carried an article about how and why Ph.D degrees were losing their sheen. In summary, its argument was basically this:

• Designed to be a programme for preparing candidates for an academic career, there is now an oversupply of Ph.D holders, with the US itself handing 64,000 doctorates every year.
• Given the low stipends/scholarships that bright Ph.D candidates are given, they have become "cheap, highly motivated and disposable labour" that is driving most of the research today, and are being taken for a ride with promises of a better future for the present struggles. The so-called "postdocs" are also victims of this infrastructure, the post itself being a result of an attempt to fix the demand-supply mismatch.
• Does having a Ph.D add value to your career? The Economist says its definitely better than a bachelor's degree, but not much value-addition over a master's - in fact you may be disadvantaged. The Ph.D skills of deep analysis, research are not what today's fast moving markets are looking for. The more qualified a candidate is, the more likely he is to be dissatisfied and de-motivated with the work at hand.

Now, here's what I think on the Economist's viewpoint:

• The demand-supply mismatch argument does not hold in India, since there is a large shortfall of qualified, well-versed academic staff today, while the number of Ph.Ds joining the pool is very small. For example, India produces just about a 100 Ph.Ds in Computer Science today. With globalization, the Indian educational institutions have to achieve higher benchmarks to maintain relevance. This explains the sprouting of new higher educational institutions like the new IITs,IIITs, IISER, and others trying to match them. These have to be manned by qualified faculty. So, we need to have more students pursuing Phds.
• Yet, we don't see too many people pursuing doctorates in India. Why? One, there is no monetary benefit for the long struggle. But that will always be the cost of pursuing anything different, hence that will never deter the really committed. Secondly, there is government apathy of which the zillions of pages have already been written. More importantly, we are not contextualizing and localizing the problems we are solving. We are solving problems defined and relevant to the West, hence neither the potential candidates nor the Indian society at large feels any affinity to the kind of problems tackled as part of the candidate's thesis. So, even the demand exists, the supply doesn't inspire enough confidence in the society. This chain of thought naturally leads to preference for the West by many wannabe candidates.
• Of course, the PhD syllabus needs to reformed. In a fast moving and dynamic world, the course needs to be more flexible. There could be intermediate exit points in the course, for instance.
  
• This seems to be part of the conservative agenda (of which the Economist is the vanguard), that wants to spend lesser and lesser on free and open education, and make the work force pay for job oriented skills. Of course, societies don't run only on markets and there should always be a place for intellectual and artistic pursuits that don't necessarily yield the market any pay-off. Society must always support such endeavours.
  

What Ph.D candidates need to do to weather this storm:

• Information is now ubiquitous, it is not a premium; hence, the days of the informed teacher being highly regarded are gone. However, with the deluge of data, the ability to analyze information, think deeply and synthesize ideas are needed more than ever. Hence Ph.D candidates have to focus more on developing their faculties in this direction. This is anyway part of their brief, but being satisfied with low-hanging fruits like being an authority of a subject will not work in future.
• They must not become super-specialists, limiting their thinking to some restricted domains of thought. After all, it is a doctorate of philosophy, and the spirit of philosophy is to think openly with no restrictions. Creativity depends a lot on cross pollination of ideas from many disciplines.
• There has to be an irreverence to authority and a commitment to be fiercely independent.
  

"Many scientists are scientists because they are afraid of life"

This quote by John Backus, the creator of FORTRAN and the Backus-Naur form inspires this post. This is what Backus had to say in his later years about science and life:

"Many scientists are scientists because they are afraid of life. It's wonderful to be creative in science be use you can do it without clashing with people and suffering the pain of relationships and making your way through the world. It's sort of this aseptic world where you can do exciting things with your faculties, and not encounter any pain. The pain in solving problems is small potatoes compared with the pain you encounter in living.
Introspection is not a scientific activity, it is not repeatable, there are no good theories about how to repeat it, what you expect to find. It's strange that by looking into yourself you really get an appreciation of the mystery of the universe. You don't by trying to find the laws of physics."

Wise words from an old hand, and they ring true. The central idea of any modern science is abstraction, the process of drawing out the essential features that describe and system, while leaving out those which are irrelevant to the goal. Through this abstraction a system is simplified, and the abstraction becomes the basis for a lot of creativity. For eg. Newton abstracted mechanical motion to precisely three laws and this spawned the modern industrial revolution and all its inventions. We now know that Newton's laws cannot explain all phenomena, neither can relativity. The uncertainty principle is a partial acceptance of the fact that everything cannot be explained through abstraction and reason. It's a messy dynamic world out there, at the level of sub atomic particles, where our clean abstractions of matter and energy, wave and particle, force and particle break down. Even our intellect is based on this kind of learning through discrimination, categorisation, and abstraction.

But abstraction is beautiful, which is why the greatest of scientists cherish these abstractions, and adjectives like 'beautiful', 'elegant', 'simple' are applied to great works of science. It is then easy to mistake the abstraction for the 'perfect' and the real for the 'impure'. In essence, one is being afraid, running away from reality - the justification of perfection is only an alibi. This belief in the 'perfection' of abstraction can become fanatical, with disastrous consequences - especially in the social sciences. And so the communist and social Darwinian ideas brought disaster to millions, as the processes of globalisation driven by a fanatical belief in the infallibility of the markets are homogenising an inherently diverse human society.

Can Eucalyptus be the "Lucene" of cloud computing?

Eucalyptus is an open source cloud platform which can be used to create private clouds. It came out of a research project at the Computer Science Department at the University of California, Santa Barbara. The good thing about it is that it has an open and modular architecture, which means it is perfect for experimenting with different kinds of innovations in cloud computing platform development.

Sounds like Lucene, the open source text search engine that with its extensible design became a vehicle for a lot of innovation in search - leading to innovations like Solr, Nutch, Hadoop, HDFS, etc.Whenever a platform is made open, and its design made modular, extensible - it becomes the hotbed of innovation. That happened with Unix, Linux, and is now happening with Android. Will Eucalyptus take cloud computing there?

PS: Cross-posted from my Peepaal blog post

Blog on organizing information

Organizing and extracting information from unstructured data is an area that excites me tremendously. I have started learning the tricks of the trade in the last couple of years, and there is still a long way to go. Dhaval shares the interest too, and we thought of writing a blog on the things we do, things we read and our learnings. We call it 'On Organizing Information', and this post should give you sense of what we intend to write about.

Green IT

There is no such thing as a free lunch. Perhaps, we thought the rules were different in cyberspace. All those free webapps, mail and storage are generating huge amounts of data and gigantic computational needs. To serve this demand, huge power guzzlers (read data centers) are springing up by the day. So, while we sit in our offices thinking that the code we write hardly makes a dent to the ecology - vast amounts of carbon is being eaten up for computation and data storage. I hardy gave these things a thought - before reading this insightful article. That's the first post on Shyamal and Sanjeev's new blog on 'Green IT'. While we can do little to reduce our dependence on this cyberworld and prevent a lot of junk data from being created, we can still look for engineering solutions to keep the problem in check. Looking forward to such solutions from Sanjeev and Shyamal.

Hackers and Painters: An unbashed recommendation

The best way to describe "Hackers and Painters": for the hacker, by the hacker and of the hacker. Paul Graham's essays are a treat to read, and they give an insight into a programmer-nerd's mindset. Graham starts with the nerd mentality in the typical American school and why nerds are different and why it is good to be a nerd. In another essay, Graham paints hacking as an art rather than a science, and does so by comparing it to, well, painting. He goes on to describe what programmers can learn from painting. Further, there are essays on the spirit of inquiry and why hackers are non-conformists and don't like authority.

The book then takes a more technical turn. One of the best essays here is about how web-based applications will change the rules of the game, load it in favour of innovators rather than the big companies and why they would make software engineering easier. Graham is quite clearly a big fan of Lisp and describes its use in his startup ViaWeb and why it is the ideal programming language of choice. There are quite a few other interesting topics he touches upon.

If that doesn't sound impressive, you can be rest assured that coming from a practitioner who has seen it all, it sounds absolutely natural, with a sense of deja vu. His writing is witty and very lucid. The arguments are well reasoned and the chain of thought is well presented. And there are quite a number of smart one-liners you can smile at. This is a case of text book essay writing (if I may use such a term). Highly recommended.

You can find many of these essays and more on Paul Graham's site.

Pune Barcamp 5

I attended my first Pune Barcamp (and my first barcamp too) this Saturday at SISCR. Basically a barcamp is an ad hoc gathering of people interesting a particular topic; but without a pre-determined formal agenda. Instead, the agenda gets decided on the day of the event, in an ad hoc manner, democratically by all the barcampers. For more read this.

There were talks and discussions on many interesting topics. The fact that it is democratic and volunteered allowed many unconventional topics like 'Knowledge Representation in Sanskrit', 'Pune e-governance and civic activism', 'e-learning', 'Save Pune traffic' and 'Why should you twitter?' to be taken up. Ofcourse, there were purely interesting tech topics like wireless security, mobile application development, Google Analytics too. And then you don't have to sit in a boring lecture, could just walk out :). However, at the barcamp there wasn't much of an incentive to walk-out. Straight, to-the-point talks kept you interested and you had a choice of talks to attend. In addition, it was a keep opportunity for networking and having a few good discussions. All-in-all a good package. Looking forward to attending more barcamps.

POCC's Startup Speed Date

Today's POCC meet was unique. The meet format allowed participants to talk to all startup registered for the meet on a one-to-one basis. It was a great platform, where I got a meet quite a lot of new people and the interesting stuff they are working on. Being interested in natural language processing and search, it was great to know companies like Alabot, 42 and Lipikaar that are doing good stuff.
As always, Navin has been liveblogging on the event. You can find briefs of the startups who registered today here.

For those familiar with POCC, it is an informal forum for investors, startups, advisors and techies in general in Pune to interact. PuneTech covers information on tech trends in Pune.

Pune Tech

Since coming back to Pune, I have been trying to get a hang of the tech scene in Pune. From the campus recruitments at IITB, my view is that Pune has a good tech ecology developing. We have had some startups working on exciting products coming to the campus. This just seems to be the right time to be in Pune.

Last week, I attended the monthly meet of the Pune Open Coffee Club (POCC). POCC is a group which seeks to promote startups, provide a platform for people with ideas to meet, seek advice, get in touch with VC, etc. and be noticed. Last week's meet was a good experience. There was a panel discussion on bootstrapping of startups with Anand Soman, Tarun Malaviya, and Shridhar Shukla and a showcase of a new startup in Pune. You can find a liveblog of the proceedings here.

A very useful resource to keep track of the tech scene in Pune is punetech.com. On the wiki, you can find links to various companies based in Pune and interest groups like the Pune Google Technology Users group. There is also a calendar, which lists all the tech events scheduled in and around Pune. All this is ofcourse community edited, so its for us techies in Pune to build on this initiative.

About Fibinacci Numbers

Discovered a few new facts about Fibonacci numbers. All I knew about Fibonacci numbers what that it is the sequence 1,1,2,3,5,8,13,21.... , where the nth number was the sum of the (n-1)th and (n-2)the Fibonacci number. Well, I have always wondered what makes these numbers special. Turns out the nth Fibonacci number can be defined as the number of ways n can be represented as a sum of 1's and 2's. That gives a combinatorial interpretation to the Fibonacci numbers, and raises them from being outcomes of a mere uninteresting addition process. I got to know about this fact during a fascinating talk at IIT, Bombay by Prof. Manjul Bhargava from Princeton. (More about that talk later).

And that's not all about the Fibonacci numbers. It turns out that any number can be represented as the sum of Fibonacci numbers. That means Fibonacci numbers can serve as a base system. However, there could be more than one way of representing the same number as a sum pf Fibonacci number. Check this to know more.

To finish this post, Fibonacci numbers were first described by the Indian linguist Hemachandra about a 100 years before Fibonacci described them and probably by Pingala in 200 B.C. too.

Information Retrieval Books

A list of good books on IR here:
http://researchonsearch.blogspot.com/2005/12/information-retrieval-textbooks.html

Compiling GIZA++

Compiling GIZA++ can be a pain, since the available source was compiled for an older version of gcc. With improved compliance provided by gcc to standard C++ in terms of template syntax and semantics, the old GIZA++ does not compile on gcc 4.2. With a lot of fixes, I was able to get GIZA++ working, but the word class creation tool mkcls proved a very tough nut to crack. Luckily, I found a link gcc 4.2 compiled source code here on the StatMT site. Hope that helps anybody looking for GIZA++ and mkcls.

Batch conversion of doc files to text files

Its kind of irritating when you need text files to run language processing applications on and you have corpora in the form of word documents. Here is a way to convert those doc files to txt files without having to open the document editor and do a 'Save As' for each corpus document. Here come Open Office macros to the rescue. You can write up a macro to save the file as a text file. The macro can then be invoked from the commandline by starting up Open Office in invisible mode. And you can wrap all this in a nice shell script to do any filtering/cleaning after saving them as text files. A note of caution: I observed that OpenOffice saves the text files asynchronously, so you the file might not be available for processing by the following script lines. Better to pause for a while, while OOffice saves the document. You can find more about this nifty timesaver here:

http://www.xml.com/pub/a/2006/01/11/from-microsoft-to-openoffice.html?page=2

Attending Conferences

Here is an interesting post Conferences: Costs and Benefits. I have never been to a conference, but having sit through many a talk, it seems only a fraction of the time spent is really useful. But I should have an experience of atleast one conference before I comment. Undoubtedly, going around asking questions is not something you can do reading conference proceedings.

You and your Research

A highly enlightening and inspiring talk by Richard Hamming about what it takes to do significant research, that I read again after a long time. Find it here.

Writing code at compile time

How do you write a C program, which at compile time allows you to input another program at the terminal. When you run the 1st program, the code that you input at the terminal should execute.

Something like:

bash#gcc -o 1 1.c
#incude
int main(void)
{
printf("Let a thousand ideas bloom :)");
}
^D
bash#./1
Let a thousand ideas bloom :)

So write the code for 1.c. Hint: Assume gcc and any UNIX variant as OS.

As it turns out, the solution is a one liner.

#include "/dev/tty"

While compiling, the macroprocessor tries to open and read from the file /dev/tty, just as it would do for any other include like stdio.h. Since /dev/tty is the terminal, you can now input the second program at the terminal. It get compiled and Bingo! you execute code written at compile time.

Where does science begin ?

Haven't you seen a lot of scientific work built on fundamental assumptions? A problem I have is in accepting unintuitive assumptions in a lot of research. Especially in a nascent field like Natural Language Processing, you find a lot of these. These are not like Euclidean axioms which seem pretty reasonable. Yet, research without such asssumptions seems to be an impossibility. The problem is then to determine the right set of axioms which serve as a basis to build the theory. Some insights I found in a Phd thesis:

Any philosophical system, any science has to start with assumptions, axioms which cannot be really proved or disproved, which are fundamentally arbitrary but hopefully convincing. In his Tractatus Logico-Philosophicus, Wittgenstein [1918] writes that the only true philosophy would be to utter proven scientific facts, to use nothing but defined symbols of a defined formalism – i.e. to renounce on metaphysics and thus on philosophy:

Die richtige Methode der Philosophie wäre eigentlich die: Nichts zu sagen, als was sich sagen lässt, also Sätze der Naturwissenschaft – also etwas, was mit Philosophie nichts zu tun hat–, und dann immer, wenn ein anderer etwas Metaphysisches sagen wollte, ihm nachzuweisen, dass er gewissen Zeichen in seinen Sätzen keine Bedeutung gegeben hat. ... (Wittgenstein 1918: 85, § 6.53)

Wittgenstein is aware that the problems with this suggestion are, however, that every definition necessitates a definition of the defining terms until we reach the unprovable maxims. If we refuse to accept these fundamental maxims, the cornerstones of meaning, we cannot state anything and are condemned to remain silent.

Wovon man nicht sprechen kann, darüber muss man schweigen. (Wittgenstein 1918: 85, § 7)

These maxims have transcendental, metaphysical quality, only they make any meaning possible and can thus instantiate our questions and answers in life.

Wir fühlen, dass, selbst wenn alle möglichen wissenschaftlichen Fragen beantwortet sind, unsere Lebensprobleme noch gar nicht berührt sind. Freilich bleibt dann eben keine Frage mehr; und eben dies ist die Antwort. (Wittgenstein 1918: 85, § 6.52)
Only the transcendental character of metaphysical philosophy can really give answers and assert meaning. If we only utter scientific proven facts we can only replace meaningless utterances with one another. E.g. in semantics we can step from language to metalanguage to meta-meta-etc.-langauge, but this does not bring us an inch closer to real meaning. On the other hand, because we cannot define the maxims we use, we remain incompetent about them nevertheless. Again, Wittgenstein’s famous quote applies:

Wovon man nicht sprechen kann, darüber muss man schweigen. (Wittgenstein 1918: 85, § 7)

We are therefore in principle disqualified from speaking, from stating anything meaningful or even “scientific”. If we accept a minimal set of maxims on which everybody agrees science seems to be possible nevertheless, as long as we can base everything on these maxims.

A New Beginning

For a long time this blog lived a dormant existence under the name 'Machine Learning Chronicle' . Over the last year, caught between baffling Gaussian tosses and intricate kernel machines which go by the fancy name of Support Vector Machines, I have not been able to write much. Ideas are not restricted to a genre or science and hence I thought of making this blog more broad-based and cover more topics in science and technology. So, here I rechristen this blog as 'Let a thousand ideas bloom!' - for ideas are the heart of science.

Over the last year, I have picked interests in information retrieval, natural language processing, cognitive sciences and the Web 2.0 phenomenon. In addition, programming, physics and space talk are always perennial favourites of mine. So that is what this place is about ... facts, thoughts and ideas.

So let me set the ball rolling ...