Thursday, May 17, 2012

Libor Scandal

WSJ Analysis Suggests Banks May Have Reported Flawed Interest Data for Libor
"Confidence in Libor matters, because the rate system plays a vital role in the global economy. Central bankers follow it closely as a barometer of the banking system's health, and to decide how much to adjust interest rates to keep their economies growing. Payments on nearly $90 trillion in dollar-denominated mortgage loans, corporate debt and financial contracts rise and fall according to Libor's movements."


Missteps on Libor Doomed Barclays's Leaders
"Mr. Diamond's downfall may have been hastened because the U.S.-born investment banker, who became chief executive at the start of 2011, had never won acceptance by Britain's political and financial establishment. When the rate-fixing scandal erupted, Mr. Diamond had few allies."


Tuesday, May 17, 2011

R Basics

R object types

Here, for more experienced users, is a list of a few of the more important object types.
Vector:
a one-dimensional array of arbitrary length. Subsets of the vector may be referenced. All elements of the vector must be of the same type--numerical, character, etc.
Matrix:
a two-dimensional array with an arbitrary number of rows and columns. Subsets of the matrix may be referenced, and individual rows and columns of the matrix may be handled as vectors. Again all elements of the matrix must be of the same type.
Array:
as a matrix, but of arbitrary dimension.
Data frame:
a set of data organised similarly to a matrix. However each column of the data frame may contain its own type of data. Columns typically correspond to variables in a statistical study, while rows correspond to observations of these variables. A data frame may be handled similarly to a matrix, and individual columns of the data frame may be handled as vectors.
Function:
see Section .
List:
an arbitrary collection of other R objects (which may include other lists).

Wednesday, April 27, 2011

R reference



I've got a R script for which I'd like to be able to supply several command-line parameters (rather than hardcode parameter values in the code itself). The script runs on Windows.
I can't find info on how to read parameters supplied on the command-line into my R script. I'd be surprised if it can't be done, so maybe I'm just not using the best keywords in my Google search...

Answer:
I made two files: exmpl.bat and exmpl.R.
  • exmpl.bat:
    set R_Script="C:\Program Files\R-3.0.2\bin\RScript.exe"
    %R_Script% exmpl.R 2010-01-28 example 100 > exmpl.batch 2>&1
    Alternatively using Rterm.exe:
    set R_TERM="C:\Program Files\R-3.0.2\bin\i386\Rterm.exe"
    %R_TERM% --no-restore --no-save --args 2010-01-28 example 100 < exmpl.R > exmpl.batch 2>&1
  • exmpl.R:
    options(echo=TRUE) # if you want see commands in output file
    args <- span=""> commandArgs(trailingOnly = TRUE)
    print(args)
    # trailingOnly=TRUE means that only your arguments are returned, check:
    # print(commandsArgs(trailingOnly=FALSE))
    
    start_date <- span=""> as.Date(args[1])
    name <- span=""> args[2]
    n <- span=""> as.integer(args[3])
    rm(args)
    
    # Some computations:
    x <- span=""> rnorm(n)
    png(paste(name,".png",sep=""))
    plot(start_date+(1L:n), x)
    dev.off()
    
    summary(x)
Save both files in the same directory and start exmpl.bat. In result you got:
  • example.png with some plot
  • exmpl.batch with all what was done
Just to add - you could add environment variable %R_Script%:
"C:\Program Files\R-3.0.2\bin\RScript.exe"
and use it in your batch scripts as %R_Script% .......
Differences between RScript and Rterm:



Econometrics in R

Thursday, April 7, 2011

My Thoughts on Thoughts on Developer Longevity

Saturday, March 5, 2011

My Thoughts on Thoughts on Developer Longevity

Davy Brion's post Thoughts on Developer Longevity has received considerable attention in the blogosphere as evidenced by the following statistics (as of this writing):

The post begins with this introduction:
How many developers over 40 do you know? For quite a few of you, the answer will be 0. For those of you who do know of one or more developers over 40: how many of them are good? If you know any good developers over 40, then do yourself a favor: start picking their brain and figure out how they keep their skills sharp. Once you've done that, do us all a favor and publish it somewhere because there really aren't that many older developers that are any good in this industry.

The post has some points which I completely agree with. For example, many organizations do "reward" or "expect" that the best developers will be managers when they "grow up." As the post states, others do move into management essentially because they find the repetition of software development less interesting or less challenging than they used to and think there are more interesting (or more effective) tactics employed in management. Bruce Eckel provides an excellent example of the latter: one would be hard-pressed to say the the author of Thinking in C++ and Thinking in Java is not "any good in this industry," but Eckel has startedchanging interests. Note that Eckel is still heavily involved in the industry, but his interests are obviously broadening. Some might argue this is a natural maturation and evolution of the best overall software engineers who understand there is more to software development than coding.

I also agree with Brion's two pieces of advice for developers who want to maintain heavily involved in coding even after they turn forty. Brion recommends "keep feeling stupid" and "question what you know and think, all the time.". I have written about similar concepts in my post Are You the Best Developer You Know? and a commenter left a similar comment on his post.

Although many of the post author's observations are similar to my own, there are also some aspects of this post that I believe are misguided or perhaps a little naive, perhaps because of the perceived youth of the author. I believe many of these are due to coming to sweeping conclusions based on limited exposure to a limited piece of the software development industry. For example, it may be true that "there really aren't that many older developers that are any good in this industry" in some of the tinker toy application development projects or in some of the many projects that dominate the blogosphere. However, in more complex domains where mathematical algorithms and deep scientific knowledge are involved, I've found that the older developers generally outshine their younger, less experienced counterparts. Even the most talented young developer has a difficult time competing with the breadth and depth of knowledge acquired over years or even decades of working in such domains.

Brion's post is an example of how a post with appropriate title and subject can lead to interesting and useful community discussion. His post is worth reading in and of itself, but the comments on his post are at least equally insightful. In the remainder of this post, I look at how some of these comments reflect some of my own observations.


Our World is Not The World

In the movies Men in Black and Men in Black II, there is a recurring theme that even the Men in Black themselves don't realize their relationship to the outer world(s). The first movie ends with a scene showing huge extraterrestrial beings placing with the planet Earth as if it is a marble. The second movie shows Agent Jay's and Agent Kay's residence (again, the Earth) simply being one of many lockers in another world's locker area. The point is that there is a much bigger world out there than the one which they live in and know. They don't know it's out there and so think the whole world is what they do know. In a sense, it's easy in any aspect in life (including development) to make sweeping judgments about the world in general or software development in particular based on our own little piece of that space. We cannot help it because we simply don't know better and often don't realize what we don't know or are missing.

It's likely that there are some projects or areas or organizations where there are few to no strong developers over forty. Having stated that, I'm aware of colleagues working on some projects and products who don't have anyone under 35 on their team because they require breadth and depth that simply aren't generally available in developers with less than 10 years experience. In either situation, it would be all too easy to make erroneous conclusions and stereotypes about developers of different age groups.

Ryan Cromwell left a short but insightful comment along these lines on that post: "I know plenty of very good 40+ developers. Be careful not to assume your world is the norm."

Achilleas Margaritis's comment on this post is also insightful in this regard:
I am 38. The developer that sits at the right position of me is 44. The developer that sits at the left position of me is 43. The guy behind him is 55. The group of developers in the end of the office space is from 35 to 60.

Please don't consider people over 40 as old. They are not. While they are not as active as those in their 20s, they have lots of experience, they know algorithms, and they can easily learn new platforms due to their experience.

In the comments section, TedT writes, "You probably don't know any because your hyper cool little startup discriminates by age so they won't hire anyone that doesn't look exactly themselves" and Nunya Biznes writes, "I don't know where you work or have worked that there are no developers over 40."


What, There's More to Life?

Ian Smith left another insightful comment on this post:
You're missing one key reason why many leave the industry before 40. After 20+ years the whole "be happy with no life" approach to keeping up-to-date (always in your own time because firms don't bother with training any more) becomes kind of lame.

I've definitely observed this. It does seem that many developers change their interests (some might say develop a better perspective) as they approach the age of forty. For many developers in this age group, new family and/or personal considerations take on a new level of importance. Time for "playing" with the latest technology can become more limited as demands for their time increase. Some developers simply want to try something new. This is not all that unusual because I think there is some satisfaction to learning more about an area in which one knows very little.


The Software Craftsman

I'm a believer in the concept of software craftsmanship. It's difficult to believe that someone can become a "craftsman" in anything in one or two years. It's far more believable that someone can become a true craftsman after many more years (seven years is often used in other industries) of experience.


The Value of History

In The Life of ReasonGeorge Santayana stated, "Those who cannot remember the past are condemned to repeat it." Many of us have heard or even quoted a slightly altered form of this: "Those who don't know history are destined to repeat it." I have found that more experienced developers often have experience in certain areas that allows them to be better judges of whether the latest language, framework, or library is simply another passing fad or something really worth future investment and consideration. Software development is a cyclic industry in which a certain approach can be in fashion for a few years, fall out of fashion for a few years, and come back into fashion again for a few years. The more experienced software developers are less likely to be easily swayed by the Lemming Effect.

Mike has an interesting comment related to this:
There is another possibility the author overlooks: younger coders' idea of what makes someone "good" is wrong. Refusing to jump on a fad until it's proven to be valuable does not make you a bad coder. Refusing to use the latest alpha release of some open source framework that was posted on freshmeat for the first time yesterday does not mean you're "behind the times". Refusing to take part in the alpha-geeking process of proving who is the techiest of all does not mean that you have no skills.


I have envied some of the instinctive skills I have witnessed some developers older than me illustrate and I have been on the other end as well when I have had younger developers ask "How did you know how to do that?" and I was able to trace it back to an experience on an earlier project.

I like baeocystin's comment on the "Thought on Developer Longevity" post: "You ask some good questions, but you should add one more- what do these intelligent, experienced ex-developers know that you don't?"


Not All Experience is Created Equal

It is my opinion that twenty years working with multiple domains, multiple problems, and multiple customer needs is far more significant and useful than twenty years doing the same thing. This has been described as the difference between doing the same thing twenty times or doing twenty different things. I extend this to learning new languages, frameworks, and tools. Although I think there's tremendous value in learning new languages and tools, my feeling is that I'd rather have a developer who knows a small number of languages well and has applied them in diverse situations over the developer who knows numerous languages but has used them all for solving the same basic or simple problem. Knowing how to write "Hello, World" in 35 different languages is not a very useful skill in and of itself. The developer who can solve complex problems in a particular language is more likely to be able to pick up a new language for solving another complex problem than the person knowing a bunch of languages only used in simple applications.


Disadvantages for More Experienced Developers

There's no question that experienced developers do face some disadvantages. While their experience can help them avoid the Lemming Effect, they can go too far the other way and not be willing to consider anything new or doing something a different way. This is covered in Brion's post. Other posts on disadvantages of experienced developers include How Experienced Developers Can Handicap a Lean Startup and Older Developers: Bad Habits Are Killing Your Career. While there may be some gross stereotyping in some of these, I do think it's worth every developer's time to always consider whether he or she is letting established habits and comfort misguide decisions.

Early in my software development career, I observed several older software developers who had been effective in days of procedural programming, but really struggled with object-oriented languages. It was not that there people were unintelligent or slow-witted. On the contrary, they had implemented some sophisticated applications despite having significantly less tools to work with than we do today. Unfortunately, they had trouble making the mental shift from procedural style to what many of us do today (object-oriented with some procedural skeletons in the closet).


What is 'Good'?

A problem with any of these types of discussions surrounds the use of subjective terms like "good." This is pointed out well by the Foredecker comment that points out problems of perception of what makes a good developer. As the comment points out, perceived outward excitement is not necessarily an indicator of a better developer.


Examples of Software Developers Over 40 Who Are At Least 'Any Good'

There are numerous examples of good software developers over the age of forty. Tim O'Brien's comment on the "Thoughts on Developer Longevity" post is: "Most of the developers I know are over 40, and they are all so good at what they do they put everyone else to shame."

I previously mentioned Bruce Eckel, who is 53 years old according to Wikipedia. The first edition of Thinking in C++ was published when he was 42 years old and he was over 40 when the first edition of Thinking in Javawas published.

According to Wikipedia, Bjarne Stroustrup was born in 1950, Tim BrayJames GoslingGrady Booch, and Tim Berners-Lee were born in 1955 (what was in the water that year?), Martin Odersky was born in 1958, Josh Bloch and Kent Beck were born in 1961, Martin Fowler was born in 1963, and Yukihiro Matsumoto was born in 1965. I think we could find room on our team for any of these individuals.


Conclusion

The post Thoughts on Developer Longevity and its associated comments are well worth the read. I agree with Brion's assertions as often, but not always, true. My major disagreement is with the percentage or ratio of good/bad developers over age forty and I think this may be due to different experiences in our careers. In the end, Brion's reasoning for the post (to determine what drives developers to remain effective after age forty) is a good way to start potentially valuable community discussion. A user going by the name "Public" perhapsanswered the question most succinctly: "I'm a 42 year old developer and my secret is this: love learning new things and spend the time to do so. There you go."

My experience has been that the most successful software development projects often are made up of a mixture of young developers who are already skilled and have even more potential with master craftsman who can guide the team through political and technological challenges they have seen many times before.

Sunday, April 25, 2010

Ten Best Chapters of Software Development Books


Saturday, April 3, 2010

Ten Best Chapters of Software Development Books

I have always had a fondness for books, both technical and non-technical. However, it seems to get increasingly difficult to read complete books. I have found that instead of reading entire books on software development, I tend to read specific chapters and portions that seem most relevant, useful, and interesting to me. In this blog post, I look at the ten chapters of software development books that I have found to be most influential or most impressive in my career so far as a software developer. These are chapters that I can enthusiastically recommend to any software developer who either wants to learn the subject of that chapter or who just wants to see what an especially well-written chapter can do.

Before getting into the chapters, it is worth noting that some of my favorite technical books are not included on this list. This is often because the entire book is great and it is difficult to isolate any one chapter that stands out from the others. For a list of great software development books, see Top 100 Best Software Engineering Books, Ever. Many of the books on this list are simply outstanding and I also like many of the "Cookbook" and "Recipes" books for practical learning, but these tend not to be the types of books with chapters on this list.

Without further adieu, here are (as of today) the ten chapters (in no particular order) that come to mind as the most impressive or influential for me from the software development books.


10. Learning Perl (Perl)

I have never cared much for Perl, but I have needed to use it off and on over the years. Because I don't use Perl regularly, I often don't remember much about it from the last time I used it. However, I have found that the first chapter of Learning Perl (Schwartz...) often provides what I need to quickly get back into writing Perl. This chapter is everything a person learning a new programming language would want in an initial chapter: a gentle introductory approach to several commonly used features of the language.


9. Java Enterprise in a Nutshell (Java)

Chapter 6 ("JNDI") of Java Enterprise in a Nutshell makes my list because it's an example of the definitive chapter on a particular subject. When I was trying to learn how to use and apply JNDI, it was difficult to find satisfactory resources. This chapter was very welcome. This is perhaps the least interesting chapter to a general audience on this list given its narrow focus and the somewhat abstracted or diminished use of JNDI in Java EE, but the chapter is representative of software development chapters on a specific subject that are the authoritative source of information on that subject.


8. Effective XML: 50 Specific Ways to Improve Your XML (XML)

The chapter from Effective XML (Elliotte Rusty Harold) making this list is not even a chapter! My favorite part of this book is its Introduction, which provides through but concise descriptions of XML terminology. This was especially important at the time of its writing to enable more precise terminology when talking about XML. I learned more from this introduction, which is available online, than I have learned from entire chapters of some XML books I have read.


7. RESTful Java with JAX-RS (Java/REST)

There are many resources (no pun intended) on Representational State Transfer (REST) that provide a high-level overview of what REST is. Chapter 2 ("Designing RESTful Services") of RESTful Java with JAX-RS (Bill Burke) goes well beyond introduction and leads the reader through the process of designing a REST-based application in a highly approachable manner. I cannot think of a better high-level description of how to apply REST (versus simply covering what REST is) than this chapter. Whether or not a developer wishes to use JAX-RS in his or her REST applications, this chapter is a useful way to begin really understanding how to design with REST.


6. Holub on Patterns: Learning Design Patterns by Looking at Code (Design Patterns)

I purchased the book Holub on Patterns: Learning Design Patterns by Looking at Code after attending Holub's presentation at SD West 2005. The book is every bit as opinionated (perhaps more so!) than Holub is in his presentations. That's why I like it so much! I especially like the first two chapters of this book. Chapter 1 ("Preliminaries: OO and Design Patterns 101") provides background information on object-oriented design and what patterns are and how to use them. This may sound basic and like nothing new, but I found many things in hear that made me re-think what I thought I knew. Primarily, I have come to agree with Holub that indiscriminate use of getters and setters is evil. Chapter 2 ("Programming with Interfaces, and a Few Creational Patterns") is also very good and covers why Holub asserts that extends is evil.

Holub believes strongly in his opinions based on a lengthy career. Although I don't always agree with everything he says and writes at the same level he does, I generally see the wisdom of what he is saying and apply similar principles (though to a less extreme degree) in my own development efforts. I like to read things that challenge my "old way of thinking," especially when they motivate improved ways of thinking and these couple chapters (with the SD West presentation) did just that.

Perhaps the most significant realization for me after reading these chapters is that objects are about the methods/behaviors and not really about the data. I had been taught that objects were structs with behaviors and this really was backwards. I especially like, and strive to follow, Holub's "prime directive" of object-oriented systems, which he defines as:

Never ask an object for information that you need to do something; rather, ask the object that has the information to do the work for you.


The get/set paradigm is really just about providing information. When an object goes way beyond providing information and does what the client needs, the need for senseless get/set methods is greatly reduced or eliminated. Specialized knowledge of how to act upon the data does not need to be in each client, but can be packaged with the data being acted upon.


5. The Practice of Programming (General Development)

The oft-cited Kernighan and Pike software development book The Practice of Programming is well-known for packing numerous useful observations from practical software development experience into a relatively (deceivingly) small book. For the software developer in a hurry, the "Epilogue" and "Appendix: Collected Rules" are especially useful. Because the "Epilogue" is only two pages and the "Appendix: Collected Rules" is only three pages, I feel like I can bend my own rules here a little and consider them together as a single chapter.

The "Epilogue" of The Practice of Programming summarizes in succinct text the underlying themes of the book such as simplicity, clarity, and interfaces. Each of the themes is summarized in a short paragraph and each theme is covered in detail earlier in the book with a chapter devoted to each theme. The "Appendix: Collected Rules" breaks down the "rules" of good software development covered in the book on a per-chapter basis as well. Between the Epilogue and this Appendix, one can quickly remember what he or she has read or can direct himself or herself to chapters in which he or she wants to learn more about that theme or set of rules.


4. Effective Java (Java)

I could have chosen several chapters from classic Java book Effective Java (Joshua Bloch) and could have filled this top ten with chapters from that book. However, the one chapter that particularly stands out to me from this information-rich book is Chapter 9 (Second Edition) on exception handling in Java.

There are many reasons for choosing this chapter in particular. One, exception handling is something that all Java developers need to understand. Two, exception handling is most effective when those throwing the exceptions and those catching and handling the exceptions use exceptions the same way. There have been numerous articles of using Java exceptions effectively, but a team that follows the principles of this chapter will need very little outside information on how to effectively apply Java exceptions.


3. Design Patterns: Elements of Reusable Object-Oriented Software (Object-Oriented Design)

For good and for bad, design patterns have left a major imprint on the software development literature. The term and its popularity started with the classic reference Design Patterns: Elements of Reusable Object-Oriented Software (GammaHelmJohnson, and Vlissides). This book was the first major work to organize a library of design patterns and established a common "pattern" for other patterns books to following (pattern name, problem, solution, and consequences).

Although the "Design Patterns" book obviously applied a name to the concept of using recurring design strategies to solve different problems, the real value it had for me as I was learning object-oriented development concepts was its introduction to the key principles behind most of the cataloged design patterns. The first chapter of this book outlines at a high-level what a design pattern is, but its real value for me was when I read the portion of this chapter regarding interfaces ("interfaces are fundamental in object-oriented systems. Objects are only known through their interfaces."), Class versus Interface Inheritance,Programming to an Interface, not an ImplementationInheritance versus Composition, and many other fundamental principles of object-oriented design and development that still seem to be ignored today.


2. Expert One-on-One J2EE Design and Development (Java EE)

It was difficult to select just one chapter from Expert One-on-One J2EE Design and Development that marked a change in how enterprise Java applications are implemented and brought what would become the Spring Framework to the attention of the world. One of the things this book did for me was to help me realize that I wasn't mistaken or missing something because I thought J2EE was more difficult than it was worth. Between this book and the 101 EJB Damnations, it was a relief to see my concerns more eloquently articulated. As I read Johnson's book, I recognized that many of the observations he was making were similar to what I had made. This made me even more interested in the topics that he covered in which I had less experience.

It was difficult to decide which chapter to feature in this list, but I initially limited it down to a choice between Chapter 4 ("Design Techniques and Coding Standards for J2EE Projects") and Chapter 6 ("Applying J2EE Technologies"). The latter, Chapter 6, features sections on deciding when to use and not use EJB and when to use and not use XML. However, I think if I have to choose, Chapter 4 is the one I will officially include in this list. That chapter is essentially an "Effective J2EE" chapter and is tremendous whether one is using Spring or not. In fact, this seemingly narrowly focused book has tons of generally useful discussion on building distributed and enterprise systems regardless of language or platform.


1. Software Craftsmanship: The New Imperative (General Development)

The book Software Craftsmanship: The New Imperative is a relatively short book and an easy read. However, I believe the general principles of software craftsmanship outlined in this book are what all developers serious about their careers should aspire to. The book also talks about how users and managers can contribute to the craft of software development, but I prefer the advice for developers because that is the area over which I have the most control. As with most of the chapters on this list, it is difficult to select just one chapter from this book. However, the very short (4 pages) Chapter 5 ("Putting People Back into Software Development") serves this purpose. In this chapter, author Pete McBreen issues this "call to arms":

We must insist that developers really know their craft before we trust them to create systems for us or with us.

Shak.blog.notes 08/20/2024

Thinking Like an Architect - InfoQ tags: blog ...