Life, universe, everything

poll, ping, pubsub, pubsubhub, pubsubhubbub

prashant — Sun, 06 Sep 2009 11:00:40 +0000

In case you are skeptical, there is a method in the madness above, in the title of the post. In tech circles it is unlikely that you haven’t heard of pubsubhubbub. In fast few months, it has been one of the top three talked about things. Other two being homomorphic encryption and Google wave[ 1.]. Coming back to current post – terms in the title indicate how you get the contents from other people. Following details become more clear, if you imagine things happening with respect to blogs, even though they are conceptually not restricted to blogs.

Polling refers to the scenario where clients keep asking the server if something new has come up. How often to ask for updates will always be a problem with polling too frequent or too infrequent, but as one reader pointed out here that one great , thing with polling is that server doesn’t have to maintain state.

Ping refers to the case where when post an article, you(or your software) also updates some popular (central) update services. Some background here.

Next is pubsub which stands for publish/subscribe , one of the earliest pitch for it was made here by Evan Henshaw-Plath and Kellan Elliott-McCrea (72 slides but worth going through). Compelling example they gave against was this – on a particular date, Friendfeed crawled Flickr 2.9 million times to get the latest photos of 45,754 users, of which 6,721 had visited Flickr in those 24 hrs and could have ‘potentially’ uploaded a photo . Note that what they proposed was not a new technology, as they point out ‘revolutionary new 20 year old technology’ . If you do it for blogs then one of the major problems with xmpp – presence data overhead, which may be as high as 60-70 % can be reduced a lot.

pubsubhub stands for publish subscribe hub and pubsubhubbub is a protocol , core of which is idea of pubsubhub. Wherein publishers(say bloggers) update the hub which(may be more than one hubs, which talk to each other) resides ‘somewhere in cloud’, as per protocol this can be push or pull as per the protocol but the next link in the chain, hub to client(say readers) it is always push model. This page is good starting point for pubsubhubbub, overview slides are good. Ever eloquent Anil Dash describes it here as pushbutton web.
Two other related reads are webhooks which is basically http callbacks. Github uses it, so does paypal for asynchronous notifications of payment in ipn.So do many others. Related concept is rsscloud which is again pubsub hub. Follow this link for details.

You might be wondering what is the point of writing all these here, there are two 1. These things are worth knowing, minimally at least and 2. This blog is pubsubhubbub enabled now via appspot hub using this wordpress plugin and feed too is pubsubhubbub enabled via feedburner, link here
–
[1]. – Good introductory read for fully homomorphic encryption is this article by Bruce Schneier and this talk is more or less only source for Google wave.

-UPDATE – This post explains how the requirement of public server for callback can be worked around for desktop clients by using xmpp gateway(for pubsubhubbub).

A bit of assembly

prashant — Sat, 29 Aug 2009 17:28:54 +0000

After reading this article about lock free buffers and seeing the use of CAS(compare and swap), I felt like posting the assembly code to do the same. Use case over there was to write a native method and call it from Java(back in 1.5 , when concurrent data structures in Java were more or less non existent). Without further ado , I’ll unleash the code onto you . First is for CAS and second is for computing GCD using Euclid’s algorithm(this one can be found in many places and tutorials as well).

Compile and run instructions gcc file_name.c ; ./a.out

Compare and Swap

  #include 
 #include 
//exchange - newvalue, comperand is old/expected value
/*
 * Function actually does the following thing - if the value at *dest is equal to oldvalue then replace it by newvalue else leave it unchanged : do all these atomically
 *
 * There are two options for return value
 *  1.is initial value of *dest and leave the burden of calling fxn to compare it with oldval
 *  2. do it over here and return 0 or 1, this should be more efficient
 * */

/* later change it into macro  */
int cas(int* dest, int oldvalue, int newvalue){
	printf("(%d,%d,%d)",*dest,oldvalue,newvalue);
	/* int cas(int dest, int oldvalue, int newvalue){ */
	/* int cas(int dest, int newvalue, int oldvalue){ */
	int result= 1;/* 1 shows that cas succeeded and 0 shows that it failed  */
	/* btw need to set cc for flag clobbering !  */
	__asm__ __volatile__(
			"movl %2, %%eax\n\t"
			"movl %3, %%ebx\n\t"
			"movl %0, %%ecx\n\t"
			"LOCK\n\t"
			"CMPXCHG %%ebx, (%%ecx)\n\t"  /* should LOCK be on the same line  */
			"jz DONE\n\t"
			"movl $0, %1\n\t "
			"DONE: \n\t"
			:"=m"(dest),"=g"(result)
			:"g" (oldvalue),"g" (newvalue),"m"(dest)
			:"%eax","%ebx","ecx","cc"
			);
	printf("(%d,%d,%d)",*dest,oldvalue,newvalue);
	return result; 
}


/* TODO
 * write another asm fxn which puts above fxn in a while loop and keep trying unless it succeeds*/

int main(){
	int a= 5,b= 6;
	int* c = (int*) malloc(sizeof(int));
	*c= 6;
	/* int c= 6; */
	printf("%d\n",cas(c,b,b));
	printf("%d\n",cas(c,b,a));
	printf("%d\n",cas(c,a,a));
	printf("%d\n",cas(c,b,b));
	*c= 6;
	/* c= 5; */
	printf("changing value of *c to %d\n",*c);
	printf("%d\n",cas(c,b,b));
	printf("%d\n",cas(c,b,a));
	printf("%d\n",cas(c,a,a));
	printf("%d\n",cas(c,a,b));
	printf("%d\n",cas(c,b,a));
	return 0;
}

Formatting notes – seems like wp syntax highlighter is adding in the end , ignore that.

GCD

#include 
int gcd( int a, int b ) {
    int result ;
    /* Compute Greatest Common Divisor using Euclid's Algorithm */
    __asm__ __volatile__ ( "movl %1, %%eax;"
                          "movl %2, %%ebx;"
                          "CONTD: cmpl $0, %%ebx;"
                          "je DONE;"
                          "xorl %%edx, %%edx;"
                          "idivl %%ebx;"
                          "movl %%ebx, %%eax;"
                          "movl %%edx, %%ebx;"
                          "jmp CONTD;"
                          "DONE: movl %%eax, %0;" : "=g" (result) : "g" (a), "g" (b)
    );

    return result ;
}

int main() {
    int first, second ;
    printf( "Enter two integers : " ) ;
    scanf( "%d%d", &first, &second );

    printf( "GCD of %d & %d is %d\n", first, second, gcd(first, second) ) ;

    return 0 ;
}

Cricketing World Awaits a New World Order

prashant — Tue, 25 Aug 2009 18:07:50 +0000

After a series spanning five tests and approximately two months, England reclaimed ashes.

Let’s talk about the winning team first – their middle order is ‘average’ if you don’t want to critical . Greame Swann’s contribution with the bat will count no less than contribution with the ball. Stuart Broad – best all rounder was non starter for the 5th test. Andrew Strauss was the only consistent batsman and Jimmy Anderson was the only consistent bowler.

Point here is not to berate the achievements of the winning team but to put in perspective the fact that despite all these, Australia lost. you’ll have to think for few seconds , if asked to name their best bowler. Right, it was BW Hilfenhaus. Hauritz outperformed everbody’s expectation but was still not good enough to be selected for two of the five tests. South Africa never became a world beater without a world class spinner , Australia can’t hope to remain one without one good enough one at least. Magician left the stage and it doesn’t look like art anymore, forget magic. Ponting stands tall among lesser mortals and Michael Clarke stands up very often, but that makes them a team with some very good players and not a very good team.
Ashes 2009 will be remembered for Andrew Flintoff’s last Ashes, Stuart Broad’s arrival as an all rounder(if he adds some more Ashes folklore to his promising start), and one of the most telling debut century of all time by Trott but it will also be remembered as the point where ifs and buts about Australia’s supremacy were finally put to rest, it is over. Don’t get me wrong they can still unearth Mike Hussey or Stuart Clark but you feel the contribution of all time greats when they go missing. Sometimes they come to remind you of themselves in IPL but still they can not be replaced. Let’s feel blessed for having seen Warne, McGrath and Gilchrist playing together and being supported by Waughs, Ponting, Hyden , Martin and Langer.

Going forward, South Africa will always remain a very good team but they need to win semifinals and be more consistent around the globe, Pakistan will remain enigmatically brilliant, Indian young guns will probably find it tough to fill the shoes of extraordinary predecessors. Sri Lanka should continue to show glimpses of art in craft. But, we may have to wait for sometime before having an all out dominant team or before a captain complains of lack of competition, as Steve Waugh did once.

On a related note – let’s welcome the coming back of Asif , he has a hell lot of potential, let’s hope to see some of that and also welcome Andy Flower in the new awatar, right you can’t keep good men down for long.

ruby one liners to create hash

prashant — Thu, 20 Aug 2009 15:45:54 +0000

Yesterday while going through one of my old programs, I found this written by me sometime back :
#begin magic hash=Hash[*CGI.unescape(raw_text).split('&').map{|x| b=x.split("=");b.push(nil) if b.size==1;b}.flatten] #end magic
To kill some of suspense let me disclose that raw_text looks like

"SUCCESS&mc_gross=10.00&protection_eligibility=Ineligible&payer_id=U7PPJJ4TSJ47E&tax=0.00&payment_date=09%3A45%3A30+Jul+10%2C+2009+PDT&payment_status=Pending"

, right it has been cut from paypal payment acknowledgment.

Above line if broken in parts reads better :

  unescaped_array= CGI.unescape(raw_text).split('&')
  unescaped_array= unescaped_array.collect{|x| b=x.split("=");b.push(nil) if b.size==1;b}
  flattened_array= unescaped_array.flatten
  hash= Hash[*flattened_array]

Let’s do individual steps in irb:

irb(main):009:0>unescaped_array= CGI.unescape(raw_text).split('&')    

=> ["SUCCESS", "mc_gross=10.00", "protection_eligibility=Ineligible", "payer_id=U7PPJJ4TSJ47E", "tax=0.00", "payment_date=09:45:30 Jul 10, 2009 PDT", "payment_status=Pending"]                                                                         

irb(main):013:0> unescaped_array= unescaped_array.map{|x| b=x.split("=");b.push(nil) if b.size==1;b}  

=> [["SUCCESS", nil], ["mc_gross", "10.00"],["protection_eligibility", "Ineligible"], ["payer_id", "U7PPJJ4TSJ47E"], ["tax", "0.00"], ["payment_date", "09:45:30 Jul 10, 2009 PDT"], ["payment_status", "Pending"]]                               

irb(main):014:0> flattened_array= unescaped_array.flatten     

=> ["SUCCESS", nil, "mc_gross", "10.00", "protection_eligibility", "Ineligible", "payer_id", "U7PPJJ4TSJ47E", "tax", "0.00", "payment_date", "09:45:30 Jul 10, 2009 PDT", "payment_status", "Pending"]
irb(main):015:0>
hash= Hash[*flattened_array]
=> {"tax"=>"0.00", "payment_status"=>"Pending", "payer_id"=>"U7PPJJ4TSJ47E", "mc_gross"=>"10.00", "SUCCESS"=>nil, "payment_date"=>"09:45:30 Jul 10, 2009 PDT", "protection_eligibility"=>"Ineligible"}

BTW, * is called splat operator in ruby

Another way to create hash from ‘array of pairs ‘ is to use inject :

hash= [[1,2],[3,4]].inject({}){|result,element| result[element.first]= result[element.last]; result}

There is one more way Write a loop, that I’ll leave as an exercise to the readers !!

Here is a bit unrelated use case of creating hash from arrays:

irb(main):005:0> [1,2,3,4,7,9].group_by{|x| x<5 ? :lesser : :greater}

=> {:lesser=>[1, 2, 3, 4], :greater=>[7, 9]}

You can do more things, basically result of the block is used as the key for that element in the resulting hash.

gmail, mutt and msmtp fix

prashant — Sun, 16 Aug 2009 19:33:29 +0000

If you use mutt and smtp to access gmail . Here is a (bad) news. Cool guys at Google again changed certificate. Oh, did you ask – how do it know it ? Simple mutt started complaining about bad certificate when trying to use msmpt, infamous ‘msmtp: TLS certificate verification failed: the certificate hasn’t got a known issuer.’ greeted me on the screen.

To cross confirm –
Just run following
$ msmtp --serverinfo --host=smtp.gmail.com --tls=on --port=587 --tls-certcheck=off
In place of old Thwate Server now you get following in issuer segment
Issuer: Common Name: Google Internet Authority Organization: Google Inc Country: US

Fortunately fix is simple, here is what you need to do on debian
# apt-get install ca-certificates # dpkg -s ca-certificates|grep Version Version: 20090814


After this just change following line in you ~/.msmtprc



tls_trust_file /certs/Thawte SSLWeb Server Roots/thawte Premium Server CA/Thawte Premium Server CA.pem



to



 tls_trust_file /usr/share/ca-certificates/mozilla/Equifax_Secure_CA.crt



Git and Awesome Survey
prashant — Sat, 15 Aug 2009 06:00:45 +0000
Here are two links for survey by  git (version control system) and awesome(window manager) community.  If you use either of these, please take out some time to fill in the questionnaire. Think of it as the simplest way to contribute back to the software you use.

Here are the links :
Git – http://www.survs.com/survey?id=2PIMZGU0&channel=Q0EKJ3NF54
Awesome – http://www.survs.com/survey?id=8BVEV3FO&channel=BH07CQ040D
Note -(As found on those pages but at the bottom   ) –

 “If you have cookies enabled, you can always submit partially filled survey, and return to your answers at later time, completing it later.”



Merging hashes in yaml conf files
prashant — Fri, 31 Jul 2009 09:12:22 +0000
YAML is  quite handy for writing configuration files. Primary advantage is that, it reads like text file. This works really well if your config file is flat(no hierarchy) and has no repetitions.

If your configurations file has repetitions then it makes sense to separate out those elements and reuse them. What I mean is this – let’s say you your config file looks like this :
 
development:
  input_location: common_input
  output_location: dev_location
  mail:
    smtp_server: your_server
    login: your_login
    password: top_secret
production:
  input_location: common_input
  output_location: dev_location
  mail:
    smtp_server: your_server
    login: your_login
    password: top_secret
Assuming above code in /tmp/test.yml here is how you can read in python and ruby

$cat readyml.py 
#!/usr/bin/env python
from pprint import pprint as pp
#in debian need to install python-yaml
from yaml import load,load_all,dump
hash= load(open('/tmp/test.yml'))
pp(hash['development'])


$ cat readyml.rb


#!/usr/bin/env ruby
require 'pp'
hash= YAML::load(File.open('/tmp/test.yml').read)
pp hash['development']
here is a handy one liner for ruby version

$ ruby  -rpp -e   'pp YAML::load(File.open("/tmp/a.yml"))["development"]'  or you can try the same in irb or python console.
Note that in the above code snippet ,everything is other than output location is same in development and production part.  This  is where yml node identifier comes to rescue. Idea is simple have a set of default values and override them at different place.

You could pull it apart as follows:

 
defaults: &defaults
  input_location: common_input
  output_location: dev_location
  mail:
    sender_name: sender
    smtp_server: your_server
    login: your_login
    password: top_secret
development:
  <<: *defaults
production:
  <<: *defaults
  output_location: prod_location


$ ruby  -rpp -e   'pp YAML::load(File.open("/tmp/a.yml"))["development"]["mail"]["login"]'

"your_login"

$



Great , it works(tm) !.

Arguably we traded some clarity for a bit of magic. Here is a small explanation : & , * and <<:  & which is  anchor tag can be understood as node identifier, * is node reference and <<: stands for hash merge.
For more details see either yaml specs or wikipedia

So far so good but there is a catch here, these hash merges are not recursive. What it means is this : let’s say you want to have different sender name for mail in two environments, you may be tempted to do the following:

 
defaults: &defaults
  input_location: common_input
  output_location: dev_location
  mail:
    sender_name: sender
    smtp_server: your_server
    login: your_login
    password: top_secret
development:
  <<: *defaults
  mail:
    sender_name: sender_dev
production:
  <<: *defaults
  output_location: prod_location
  mail:
    sender_name: sender_prod
Lets check



$ ruby  -rpp -e   'pp YAML::load(File.open("/tmp/a.yml"))["development"]["mail"]["login"]'

nil

$


Oops, something went wrong, problem as mentioned above is that the hash merge is not recursive and while merging it replaced mail of default by mail of production which has only one key. Solution/work around is to unroll one more level:

 
common_settings: &common_settings
input_location: common_input
output_location: dev_location
mail_defaults: &mail_defaults
  sender_name: sender
  smtp_server: your_server
  login: your_login
  password: top_secret

defaults: &defaults
  <<: *common_settings
  mail:
    <<: *mail_defaults
development:
  <<: *defaults
production:
  <<: *defaults
  mail:
    <<: *mail_defaults
    sender_name: sender_prod

Lets check again



$ ruby  -rpp -e   'pp YAML::load(File.open("/tmp/a.yml"))["development"]["mail"]["login"]'

"your_login"

$



Did you say you have one more level of nesting, well you can definitely unroll one more level, but then it becomes a mess. So, if you are not trying to write solution to towers of hanoi in a conf file, it is better to restucture conf file than digging into yaml or something else. But that is your call anyway.



Great programmers find simple solutions to complex problems
prashant — Tue, 30 Jun 2009 11:46:53 +0000
After a long time I could find the source for following quote-
Junior programmers create simple solutions to simple problems. Senior programmers create complex solutions to complex problems. Great programmers find simple solutions to complex problems.
It is from an essay called  Beautiful Software by Charles Connell.
Another favorite one of mine is The Zen of Python -
Beautiful is better than ugly.

Explicit is better than implicit.

Simple is better than complex.

Complex is better than complicated.

Flat is better than nested.

Sparse is better than dense.

Readability counts.

Special cases aren’t special enough to break the rules.

Although practicality beats purity.

Errors should never pass silently.

Unless explicitly silenced.

In the face of ambiguity, refuse the temptation to guess.

There should be one– and preferably only one –obvious way to do it.

Although that way may not be obvious at first unless you’re Dutch.

Now is better than never.

Although never is often better than *right* now.

If the implementation is hard to explain, it’s a bad idea.

If the implementation is easy to explain, it may be a good idea.

Namespaces are one honking great idea — let’s do more of those!

————————————————————————–

and then there is classic Tao of Programming 
—
credits as on the linked site-
Translated by Geoffrey James
Transcribed by Duke Hillard
Transmitted by Anupam Trivedi, Sajitha Tampi, and Meghshyam Jagannath
Re-html-ized and edited by Kragen Sittler
—-

Book 1 – The Silent Void
Thus spake the master programmer:
“When you have learned to snatch the error code from the trap frame, it

will be time for you to leave.”
1.1
Something mysterious is formed, born in the silent void. Waiting alone and

unmoving, it is at once still and yet in constant motion. It is the source

of all programs. I do not know its name, so I will call it the Tao of

Programming.
If the Tao is great, then the operating system is great. If the operating

system is great, then the compiler is great. If the compiler is great,

then the application is great. The user is pleased and there exists

harmony in the world.
The Tao of Programming flows far away and returns on the wind of morning.
1.2
The Tao gave birth to machine language. Machine language gave birth to the

assembler.
The assembler gave birth to the compiler. Now there are ten thousand

languages.
Each language has its purpose, however humble. Each language expresses the

Yin and Yang of software. Each language has its place within the Tao.
But do not program in COBOL if you can avoid it.
1.3
In the beginning was the Tao. The Tao gave birth to Space and Time.

Therefore Space and Time are Yin and Yang of programming.
Programmers that do not comprehend the Tao are always running out of time

and space for their programs. Programmers that comprehend the Tao always

have enough time and space to accomplish their goals.
How could it be otherwise?
1.4
The wise programmer is told about Tao and follows it. The average

programmer is told about Tao and searches for it. The foolish programmer

is told about Tao and laughs at it.
If it were not for laughter, there would be no Tao.
The highest sounds are hardest to hear.

Going forward is a way to retreat.

Great talent shows itself late in life.

Even a perfect program still has bugs.
————————————————————————–
Book 2 – The Ancient Masters
Thus spake the master programmer:
“After three days without programming, life becomes meaningless.”
2.1
The programmers of old were mysterious and profound. We cannot fathom

their thoughts, so all we do is describe their appearance.
Aware, like a fox crossing the water. Alert, like a general on the

battlefield. Kind, like a hostess greeting her guests. Simple, like

uncarved blocks of wood. Opaque, like black pools in darkened caves.
Who can tell the secrets of their hearts and minds?
The answer exists only in Tao.
2.2
Grand Master Turing once dreamed that he was a machine. When he awoke he

exclaimed:
“I don’t know whether I am Turing dreaming that I am a machine, or a

machine dreaming that I am Turing!”
2.3
A programmer from a very large computer company went to a software

conference and then returned to report to his manager, saying: “What sort

of programmers work for other companies? They behaved badly and were

unconcerned with appearances. Their hair was long and unkempt and their

clothes were wrinkled and old. They crashed our hospitality suite and they

made rude noises during my presentation.”
The manager said: “I should have never sent you to the conference. Those

programmers live beyond the physical world. They consider life absurd, an

accidental coincidence. They come and go without knowing limitations.

Without a care, they live only for their programs. Why should they bother

with social conventions?
“They are alive within the Tao.”
2.4
A novice asked the Master: “Here is a programmer that never designs,

documents or tests his programs. Yet all who know him consider him one of

the best programmers in the world. Why is this?”
The Master replies: “That programmer has mastered the Tao. He has gone

beyond the need for design; he does not become angry when the system

crashes, but accepts the universe without concern. He has gone beyond the

need for documentation; he no longer cares if anyone else sees his code.

He has gone beyond the need for testing; each of his programs are perfect

within themselves, serene and elegant, their purpose self-evident. Truly,

he has entered the mystery of Tao.”
————————————————————————–
Book 3 – Design
Thus spake the master programmer:
“When the program is being tested, it is too late to make design

changes.”
3.1
There once was a man who went to a computer trade show. Each day as he

entered, the man told the guard at the door:
“I am a great thief, renowned for my feats of shoplifting. Be

forewarned, for this trade show shall not escape unplundered.”
This speech disturbed the guard greatly, because there were millions of

dollars of computer equipment inside, so he watched the man carefully. But

the man merely wandered from booth to booth, humming quietly to himself.
When the man left, the guard took him aside and searched his clothes, but

nothing was to be found.
On the next day of the trade show, the man returned and chided the guard

saying: “I escaped with a vast booty yesterday, but today will be even

better.” So the guard watched him ever more closely, but to no avail.
On the final day of the trade show, the guard could restrain his curiosity

no longer. “Sir Thief,” he said, “I am so perplexed, I cannot live in

peace. Please enlighten me. What is it that you are stealing?”
The man smiled. “I am stealing ideas,” he said.
3.2
There once was a master programmer who wrote unstructured programs. A

novice programmer, seeking to imitate him, also began to write

unstructured programs. When the novice asked the master to evaluate his

progress, the master criticized him for writing unstructured programs,

saying, “What is appropriate for the master is not appropriate for the

novice. You must understand the Tao before transcending structure.”
3.3
There was once a programmer who was attached to the court of the warlord

of Wu. The warlord asked the programmer: “Which is easier to design: an

accounting package or an operating system?”
“An operating system,” replied the programmer.
The warlord uttered an exclamation of disbelief. “Surely an accounting

package is trivial next to the complexity of an operating system,” he

said.
“Not so,” said the programmer, “when designing an accounting package,

the programmer operates as a mediator between people having different

ideas: how it must operate, how its reports must appear, and how it must

conform to the tax laws. By contrast, an operating system is not limited

by outside appearances. When designing an operating system, the programmer

seeks the simplest harmony between machine and ideas. This is why an

operating system is easier to design.”
The warlord of Wu nodded and smiled. “That is all good and well, but

which is easier to debug?”
The programmer made no reply.
3.4
A manager went to the master programmer and showed him the requirements

document for a new application. The manager asked the master: “How long

will it take to design this system if I assign five programmers to it?”
“It will take one year,” said the master promptly.
“But we need this system immediately or even sooner! How long will it

take if I assign ten programmers to it?”
The master programmer frowned. “In that case, it will take two years.”
“And what if I assign a hundred programmers to it?”
The master programmer shrugged. “Then the design will never be

completed,” he said.
————————————————————————–
Book 4 – Coding
Thus spake the master programmer:
“A well-written program is its own heaven; a poorly-written program is

its own hell.”
4.1
A program should be light and agile, its subroutines connected like a

string of pearls. The spirit and intent of the program should be retained

throughout. There should be neither too little or too much, neither

needless loops nor useless variables, neither lack of structure nor

overwhelming rigidity.
A program should follow the `Law of Least Astonishment’. What is this law?

It is simply that the program should always respond to the user in the way

that astonishes him least.
A program, no matter how complex, should act as a single unit. The program

should be directed by the logic within rather than by outward appearances.
If the program fails in these requirements, it will be in a state of

disorder and confusion. The only way to correct this is to rewrite the

program.
4.2
A novice asked the master: “I have a program that sometime runs and

sometimes aborts. I have followed the rules of programming, yet I am

totally baffled. What is the reason for this?”
The master replied: “You are confused because you do not understand Tao.

Only a fool expects rational behavior from his fellow humans. Why do you

expect it from a machine that humans have constructed? Computers simulate

determinism; only Tao is perfect.
“The rules of programming are transitory; only Tao is eternal. Therefore

you must contemplate Tao before you receive enlightenment.”
“But how will I know when I have received enlightenment?” asked the

novice.
“Your program will then run correctly,” replied the master.
4.3
A master was explaining the nature of Tao of to one of his novices. “The

Tao is embodied in all software – regardless of how insignificant,” said

the master.
“Is the Tao in a hand-held calculator?” asked the novice.
“It is,” came the reply.
“Is the Tao in a video game?” continued the novice.
“It is even in a video game,” said the master.
“And is the Tao in the DOS for a personal computer?”
The master coughed and shifted his position slightly. “The lesson is over

for today,” he said.
4.4
Prince Wang’s programmer was coding software. His fingers danced upon the

keyboard. The program compiled without an error message, and the program

ran like a gentle wind.
“Excellent!” the Prince exclaimed, “Your technique is faultless!”
“Technique?” said the programmer turning from his terminal, “What I

follow is Tao – beyond all techniques! When I first began to program I

would see before me the whole problem in one mass. After three years I no

longer saw this mass. Instead, I used subroutines. But now I see nothing.

My whole being exists in a formless void. My senses are idle. My spirit,

free to work without plan, follows its own instinct. In short, my program

writes itself. True, sometimes there are difficult problems. I see them

coming, I slow down, I watch silently. Then I change a single line of code

and the difficulties vanish like puffs of idle smoke. I then compile the

program. I sit still and let the joy of the work fill my being. I close my

eyes for a moment and then log off.”
Prince Wang said, “Would that all of my programmers were as wise!”
————————————————————————–
Book 5 – Maintenance
Thus spake the master programmer:
“Though a program be but three lines long, someday it will have to be

maintained.”
5.1
A well-used door needs no oil on its hinges.

A swift-flowing stream does not grow stagnant.

Neither sound nor thoughts can travel through a vacuum.

Software rots if not used.
These are great mysteries.
5.2
A manager asked a programmer how long it would take him to finish the

program on which he was working. “It will be finished tomorrow,” the

programmer promptly replied.
“I think you are being unrealistic,” said the manager, “Truthfully, how

long will it take?”
The programmer thought for a moment. “I have some features that I wish to

add. This will take at least two weeks,” he finally said.
“Even that is too much to expect,” insisted the manager, “I will be

satisfied if you simply tell me when the program is complete.”
The programmer agreed to this.
Several years later, the manager retired. On the way to his retirement

luncheon, he discovered the programmer asleep at his terminal. He had been

programming all night.
5.3
A novice programmer was once assigned to code a simple financial package.
The novice worked furiously for many days, but when his master reviewed

his program, he discovered that it contained a screen editor, a set of

generalized graphics routines, an artificial intelligence interface, but

not the slightest mention of anything financial.
When the master asked about this, the novice became indignant. “Don’t be

so impatient,” he said, “I’ll put in the financial stuff eventually.”
5.4
Does a good farmer neglect a crop he has planted?

Does a good teacher overlook even the most humble student?

Does a good father allow a single child to starve?

Does a good programmer refuse to maintain his code?
————————————————————————–
Book 6 – Management
Thus spake the master programmer:
“Let the programmers be many and the managers few – then all will be

productive.”
6.1
When managers hold endless meetings, the programmers write games. When

accountants talk of quarterly profits, the development budget is about to

be cut. When senior scientists talk blue sky, the clouds are about to roll

in.
Truly, this is not the Tao of Programming.
When managers make commitments, game programs are ignored. When

accountants make long-range plans, harmony and order are about to be

restored. When senior scientists address the problems at hand, the

problems will soon be solved.
Truly, this is the Tao of Programming.
6.2
Why are programmers non-productive?

Because their time is wasted in meetings.
Why are programmers rebellious?

Because the management interferes too much.
Why are the programmers resigning one by one?

Because they are burnt out.
Having worked for poor management, they no longer value their jobs.
6.3
A manager was about to be fired, but a programmer who worked for him

invented a new program that became popular and sold well. As a result, the

manager retained his job.
The manager tried to give the programmer a bonus, but the programmer

refused it, saying, “I wrote the program because I thought it was an

interesting concept, and thus I expect no reward.”
The manager upon hearing this remarked, “This programmer, though he holds

a position of small esteem, understands well the proper duty of an

employee. Let us promote him to the exalted position of management

consultant!”
But when told this, the programmer once more refused, saying, “I exist so

that I can program. If I were promoted, I would do nothing but waste

everyone’s time. Can I go now? I have a program that I’m working on.”
6.4
A manager went to his programmers and told them: “As regards to your work

hours: you are going to have to come in at nine in the morning and leave

at five in the afternoon.” At this, all of them became angry and several

resigned on the spot.
So the manager said: “All right, in that case you may set your own

working hours, as long as you finish your projects on schedule.” The

programmers, now satisfied, began to come in at noon and work to the wee

hours of the morning.
————————————————————————–
Book 7 – Corporate Wisdom
Thus spake the master programmer:
“You can demonstrate a program for a corporate executive, but you can’t

make him computer literate.”
7.1
A novice asked the master: “In the east there is a great tree-structure

that men call `Corporate Headquarters’. It is bloated out of shape with

vice presidents and accountants. It issues a multitude of memos, each

saying `Go, Hence!’ or `Go, Hither!’ and nobody knows what is meant. Every

year new names are put onto the branches, but all to no avail. How can

such an unnatural entity be?”
The master replied: “You perceive this immense structure and are

disturbed that it has no rational purpose. Can you not take amusement from

its endless gyrations? Do you not enjoy the untroubled ease of programming

beneath its sheltering branches? Why are you bothered by its

uselessness?”
7.2
In the east there is a shark which is larger than all other fish. It

changes into a bird whose wings are like clouds filling the sky. When this

bird moves across the land, it brings a message from Corporate

Headquarters. This message it drops into the midst of the programmers,

like a seagull making its mark upon the beach. Then the bird mounts on the

wind and, with the blue sky at its back, returns home.
The novice programmer stares in wonder at the bird, for he understands it

not. The average programmer dreads the coming of the bird, for he fears

its message. The master programmer continues to work at his terminal, for

he does not know that the bird has come and gone.
7.3
The Magician of the Ivory Tower brought his latest invention for the

master programmer to examine. The magician wheeled a large black box into

the master’s office while the master waited in silence.
“This is an integrated, distributed, general-purpose workstation,” began

the magician, “ergonomically designed with a proprietary operating

system, sixth generation languages, and multiple state of the art user

interfaces. It took my assistants several hundred man years to construct.

Is it not amazing?”
The master raised his eyebrows slightly. “It is indeed amazing,” he

said.
“Corporate Headquarters has commanded,” continued the magician, “that

everyone use this workstation as a platform for new programs. Do you agree

to this?”
“Certainly,” replied the master, “I will have it transported to the

data center immediately!” And the magician returned to his tower, well

pleased.
Several days later, a novice wandered into the office of the master

programmer and said, “I cannot find the listing for my new program. Do

you know where it might be?”
“Yes,” replied the master, “the listings are stacked on the platform in

the data center.”
7.4
The master programmer moves from program to program without fear. No

change in management can harm him. He will not be fired, even if the

project is cancelled. Why is this? He is filled with Tao.
————————————————————————–
Book 8 – Hardware and Software
Thus spake the master programmer:
“Without the wind, the grass does not move. Without software, hardware is

useless.”
8.1
A novice asked the master: “I perceive that one computer company is much

larger than all others. It towers above its competition like a giant among

dwarfs. Any one of its divisions could comprise an entire business. Why is

this so?”
The master replied, “Why do you ask such foolish questions? That company

is large because it is large. If it only made hardware, nobody would buy

it. If it only made software, nobody would use it. If it only maintained

systems, people would treat it like a servant. But because it combines all

of these things, people think it one of the gods! By not seeking to

strive, it conquers without effort.”
8.2
A master programmer passed a novice programmer one day. The master noted

the novice’s preoccupation with a hand-held computer game. “Excuse me,”

he said, “may I examine it?”
The novice bolted to attention and handed the device to the master. “I

see that the device claims to have three levels of play: Easy, Medium, and

Hard,” said the master. “Yet every such device has another level of

play, where the device seeks not to conquer the human, nor to be conquered

by the human.”
“Pray, great master,” implored the novice, “how does one find this

mysterious setting?”
The master dropped the device to the ground and crushed it underfoot. And

suddenly the novice was enlightened.
8.3
There was once a programmer who worked upon microprocessors. “Look at how

well off I am here,” he said to a mainframe programmer who came to visit,

“I have my own operating system and file storage device. I do not have to

share my resources with anyone. The software is self- consistent and

easy-to-use. Why do you not quit your present job and join me here?”
The mainframe programmer then began to describe his system to his friend,

saying “The mainframe sits like an ancient sage meditating in the midst

of the data center. Its disk drives lie end-to-end like a great ocean of

machinery. The software is as multifaceted as a diamond, and as convoluted

as a primeval jungle. The programs, each unique, move through the system

like a swift-flowing river. That is why I am happy where I am.”
The microcomputer programmer, upon hearing this, fell silent. But the two

programmers remained friends until the end of their days.
8.4
Hardware met Software on the road to Changtse. Software said: “You are

Yin and I am Yang. If we travel together we will become famous and earn

vast sums of money.” And so the set forth together, thinking to conquer

the world.
Presently they met Firmware, who was dressed in tattered rags and hobbled

along propped on a thorny stick. Firmware said to them: “The Tao lies

beyond Yin and Yang. It is silent and still as a pool of water. It does

not seek fame, therefore nobody knows its presence. It does not seek

fortune, for it is complete within itself. It exists beyond space and

time.”
Software and Hardware, ashamed, returned to their homes.
————————————————————————–
Book 9 – Epilogue
Thus spake the master programmer:
“It is time for you to leave.”
————————————————————————–

If you liked these, you may also like to read worse is better



A bit of shell redirection
prashant — Sat, 09 May 2009 19:07:18 +0000
Here is how we normally do shell redirection

 $ ./pgm.sh args >out.txt 2>err.txt 

I wanted to modify it a bit and run as follows

 $ ./pgm.sh args 

with the requirement that  output and error should go to some filename computed inside pgm.sh based on args. One illustrative case could be when date is part of args. So you would like stdout to go to say /your/directory/pgm_out_YYYYMMDD.txt 1
The problem with standard way of redirecting  N>file.txt i.e, associating file descriptor N to file.txt , is that it works only for the newly forked process and not for the current  process.

so

$ echo hi 1>out.txt ;  echo hii  will send hi to out.txt but will print hii to stdout.2
This is where exec comes to our rescue. If we add  exec 1>somefile.txt  then output from rest of the script will go to somefile.txt



$ cat test.sh

#!/usr/bin/env bash

exec 1>out.txt

echo hi

echo hii

$./test.sh will redirect hi as well as hii to out.txt
Similarly to redirect stdout as well as stderr we’ll do something like this

 cat test2.sh

exec 1>out.txt

exec 2>err.txt

echo out text

echo 1>&2 err text

somenoneexitent command

ls -ld /tmp



Now coming back to original point of redirecting to some file from inside the shell, let’s say program computed the filename in some variable  OUTFILE, we could have just done  exec 1>$OUTFILE
That solves the current problem. But you may like to go through following example which achieves ‘random access’ of file in shell script.  Example is  from  here

echo 1234567890 > File    # Write string to "File".

exec 3<> File             # Open "File" and assign fd 3 to it.

read -n 4 <&3             # Read only 4 characters.

echo -n . >&3             # Write a decimal point there.

exec 3>&-                 # Close fd 3.

cat File                  # ==> 1234.67890



With comments, this code is self explanatory.

–

1 It can also be done by  $ ./pgm.sh args >pgm_out`date +%Y%m%d`  but idea is to generate this file name based on some logic in program itself.

2  1 in 1>out.txt is redundant but it clarifies here that we are redirecting fd 1



Sony VAIO  review
prashant — Tue, 28 Apr 2009 19:06:21 +0000
A month back I  bought a vaio NS25G and have been planning to write this post since then. This post is  primarily notes to self and review of the same. Let’s get off the mark quickly -
Post has two parts -
I. Review  of hardware

II. Installing debian and things to setup
I. Review  of hardware 
Since I am not a VAIO marketing guy,  I’ll start other way round -
Why you may not like to buy this laptop :

No bluetooth : believe me, in 2009 sony sells laptop without bluetooth. Sure you can use usb stuff but …
Screen size is 15.4 inches :  if you travel a lot then this laptop may not be very handy and also 2.9kg laptop is not exactly the lightest around.
No HDMI output.
No  separate graphics card and the likes.(and no dedicated video memory

Now , why you may like to buy it : 

This laptop is a damn good desktop. What I mean is this if your requirement is like me – mostly writing programs and trying to use it 25 out of 24 hrs, then with a big screen , extremely comfortable keyboard layout with spacious palm rest and good compute power , it makes a very good combination.
It L2 cache is 2MB and processor/memory system bus are 800MHz. I don’t play games so can’t say about that but for anything else but a core 2 duo processor with 2GHz freq is good, this is more than enough.
 From my usage, I see processor mostly clocks at 1.2GHz, and memory usage ~500MB. That just reinforces previous point.
Battery backup is good, ~2  hrs is what I get.

Just one more thing here – being  Sony,  it will cost you a bit than other laptops with similar configuration,  but that is a call u have to make.
II.  Configuring linux (to run out of box ! ) :  

It came pre installed with Vista( I think home premium or some fancy name like that). Dungara booted once and created another partition of 150 GB.  Debian testing is what I always install and then up(down?)grade to ‘unstable’ sid.
Things which worked without any problem:

 X -  unsurprisingly X server had not problem with starting or resolution.  After one of the upgrades X refused to start though,  but  ‘X -configure’ generated a conf file which worked pretty well.
 Audio -  Audio output and external mic work fine. kernel version 2.6.29-1-686 #1 SMP and alsa version  1.0.17. This is what you get from

 $ lspci  -v |grep Audio 00:1b.0 Audio device: Intel Corporation 82801I (ICH9 Family) HD Audio Con1.0.17troller (rev 03)
Video cam – just works
Hibernate to disk and back – no problem
Wireless -  in short  – just works.

 $ lspci |grep -i wireless

03:13:55 04:00.0 Network controller: Atheros Communications Inc. AR928X Wireless Network Adapter (PCI-Express) (rev 01) .

I think kernel 2.6..27 onwards it works with the stock kernel source. You dont have to install ndiswrapper or things like t

hat. As a note, please say good buy gnome-network-manager and welcome wicd, it is just a gui layer but behind the scene they have fixed lot of config stuff like wpasupplient and more.

What doesn’t work -

Somehow internal mic refuses to work    External mic works fine though.  Latest alsa sources did not compile cleanly on debian, so for the  time being  I bought external Internal mic, will try to dig into it sometime in future though.

Few more misc hints for deiban:

Package for devnagri font- ttf-indic-fonts
Package for flash player – flashplugin-nonfree
Entry in sources.list for mplayer and likes – deb http://www.debian-multimedia.org sid main
Entry in sources.list for e17 – deb http://debian.alphagemini.org/ unstable main
Entry in sources.list for skype – http://download.skype.com/linux/repos/debian/ stable non-free

oneliner for increasing and decreasing the brightness -



[root]prashant@vaio09:12 PM$ cat bin/incbrightness.sh

#!/usr/bin/env bash

echo $[1+`cat /sys/class/backlight/acpi_video0/brightness `] >/sys/class/backlight/acpi_video0/brightness



you’ll have to put this in sudo though.

and to decrease change + to – and … you get the idea, right !
A bit more, if you use enlightenment(e17) here are few more hints -
[enlightenment key bindidings]
You can use following key code and mapping
mute :

KEY="Keycode-160"   ACTION="exec" PARAMS="/usr/bin/amixer -q set Master toggle"

hibernate :

KEY="Keycode-165"   ACTION="exec" PARAMS="sudo /usr/sbin/hibernate-disk"

volume decrease :

KEY="Keycode-174"   ACTION="exec" PARAMS="/usr/bin/amixer -q set Master 2-"

volume increase:

KEY="Keycode-176"   ACTION="exec" PARAMS="/usr/bin/amixer -q set Master 2+"

increase brightness :

KEY="Keycode-212"   ACTION="exec" PARAMS="sudo /root/bin/incbrightness.sh" 

decrease brightness

KEY="Keycode-101" ACTION="exec" PARAMS="sudo /root/bin/decbrightness.sh"