Several who have read my blog contact me concerning reference materials and textbooks that I would recommend. I must admit that I am a biblio-maniac. I am unable to go into a book store especially a used bookstore and walk out empty handed. Even in this day and age of iPads I still love the visceral aspects of the printed form. Call me a luddite. That said I do have have somewhat of an interesting per-view on the world of books.
For the software world the people that are doing the creating are not writing books per se. Thus if your reading books your not necessarily creating anything novel. Let me be more specific. Many use books to find an answer. Many use books as a catalyst or reference. I fall into the latter. There are no “give me a shipping product” books – no “what is the answer?” - for some of the stuff that we create. Yes there are several ‘cookbooks’ and recipes for say learning a language or a new framework but there are no books that will provide answers to production level novel solutions. As a very succinct example I was having a discussion in NYC with a very respected Executive Engineering Director and we both decided that there were no books that dealt specifically with hiring great talent in the tech world. Why? Because most are not going to tell you exactly what it really takes to hire great people and to build great teams. That said few are going to explain how they came up with great coding feats – they will explain it after the fact for reference see Coders At Work.
So without further “waxin all philosphical‘” here is a laundry list of books that I have found useful over the years. I will try and categorize them according to subject matter. Its by no means comprehensive. I will also utilize this same list format in the future. I would love to hear feedback or your thoughts – please share your list, if you are so inclined – after the fact.
Machine Learning, Data Mining and Knowledge Discovery
This book is phenomenal and is at the very for front of issues we are all trying to solve in the areas of large scale learning systems. The book contains everything from parallelization of Support Vector Machines to details of how one designs hardware architectures for the next round of highly scalable machine learning algorithms. Note this is not an introductory text. Also the book takes a very interesting view in that it is written mainly by the best in the field.
This is a seminal text in the areas of Machine Learning. There is enough verbiage if your not a mathematician and enough algorithmic pseudo code to follow the development of the explanations. I find myself going back to this book over and over. Covers most everything from Bayes to Boosting. It also has updated chapters on his web page. I also like this writing style.
Some purest might not like this book but I think it is a great read because it discusses why we actually need to consider a paying customer. The book goes deep enough in the explanations and marries them to great scenarios for business models. It also discusses in a ‘mind map manner’ how to design systems and algorithms. The text also has a great bibliography.
This is a great book that gets your toes in the water for differing aspects of “DataScience”. Basically statistics and machine learning for data. The book has examples for simple crawlers, Bayes Theorem, Adaptive Neural Networks, Similarity Functions, Distance Metrics, and workhorse algorithms such as Support Vector Machines and Non-Negative Matrix Factorization. Written in a pragmatic fashion. Code in Python is included on the author’s website.
Seminal book on the subject. Spanning is the word I think of when reading this tome. The book covers the usual suspects but goes on to a ‘meta-level’ to discuss varied fields of Logic, Qualitative Physics, Spatial Reasoning, Psychological foundations and fundamental mathematics. The appendix and bibliography are worth the price of entry (expensive) for this oracle.
I am a proponent of the ‘get it and go’ format. This lives up to the title. It is a good introductory text. It starts off with a good introduction to Genetic Algorithms, not to be confused with Evolutionary Computing and proceeds to Learning systems and deeper issues as Non-Stationary classification systems, multi-variate parameter selection as well as a chapter on applications.
This book takes a novel approach that have an online presence and a hard text reference. Stephen Marsland updates the code and book online and provides access much like the O’Reilly series. Of particular interest is the format for review of such areas as probability, eigenvaules and fundamental algorithms. The book also provides code examples mainly in the areas of Python. This book takes exactly that approach, with each topic being presented both mathematically and in Python code using the Numpy and Scipy libraries. As most know SciPy and Numpy resemble Matlab and is sufficiently high level that the book code
examples read like pseudocode. SciKits would also be a good addition.
Semantics and Natural Language Processing
With the same eye towards ‘get in and get done’ Seagram wrote this because there was no middle ground for Semantics or if there were such a text it was voluminous. Great introductory text on Semantics with succinct code examples in Python. The text emphasizes that you are not constrained to using data from only one database or server, but when the data is organized and stored in a standardized semantic format such as RDF, it may be made available for reading by anyone (if desired) without any risk of it being overwritten by using a query language like SPARQL instead of the better known SQL. Once again complete code available via the InterWebz.
This book is a larger text that spans the areas of Semantics: FOAF, Reasoners, SPARQL, Linked Data, Microformats without getting too far in the weeds with theoretic gymnastics (read axiomatic proofs etc). Written in mainly in JAVA. The discussion of Semantics from a ‘non ivory tower’ standpoint is worth the purchase. Then again it has been said in order to program semantics one has to believe semantics exist.
People often confuse Semantics with Natural Language Processing. While the two are close cousins you can do one without the other and visa versa. Historically NLTKas it is now known in the industry is becoming the “goto” Python Library for performing textual gymnastics. This book is a wonderful introduction to Python, Text Analytics and analysis of grammer. For example entity extraction which historically is extremely difficult is:
entities = nltk.chunk.ne_chunk(tagged)
I mean (pun intended) how nice is that?
This is at a much lower level and assumes that you need to do perform basic NLP and is more text analytics based than extracting meaning, linguistic and grammatical behaviors. One novel aspect to this book is the addition of creation of a corpora reader via MongoDb. The book even addresses nuances like FileLocking during corpora edits. It uses PyMongo as the front end interface with Python examples. The examples go on the look at various aspects of NLTK and NoSQL architectures.
Network and Dynamic Systems
At the moment this is one of the tomes that I am immersed within. The book has a good introduction on Graph Theory but then quickly heads into territory such as calculations of Social Capital with emphasis on Closure within Networks. The text also does a grand job of tying (pun intended) Graph Theory and Game Theoretic applications with emphasis on arbitrage environments. The chapter on diffusion in networks is worth the price of purchase. For anyone performing work in the area of Social Networks this text is a must.
This book is a great mix of statistics, linear algebra and game theory as applied to social networks. The economic emphasis is particularly cogent. The detail and depth is world class. The analysis of Stochastic and Random graph models is of particular interest for anyone dealing with adoption models in network environments. Further the mathematical analysis of imitation in social networks is especially enjoyable. This is also one of the few text that I have seen that tie together Fat Tail, Long Tail and Poisson Distribution analysis.
This is an older text published in 1979. Funny how those “maths” do not change much – eh? I have owned this book since I have been involved with creating software systems. The book while not “out of print” is commanding extremely large sums of money. The objective as stated in the preface is: “to help one develop the ability to analyze real dynamic phenomena and dynamic systems.” Seems very appropriate in this day and age of online behavior. The book starts off with a review of differential equations, linear algebra and state space equations. Then text then goes on to address both mechanical and electrical constructs with applications with tools such as markov models and control systems with z-transform theory. You read this book and honestly you realize not much as changed with the InterWebz.
Years ago when I was working on my Phd in Applied Science we had several levels of Numerical Analysis. I love the aspects of Lagrange Polynomials, L2 Norms, Splines and the like. This book covers them all and has a really cool glossary and index of defined terms for that moment when you literally forgot what a term means because your cross eyed.
This is/was a text for a class at c on - Probability Random Variables and Stochastic Processes. The book is written by Athanasios Papoulis who is a legend in signal processing. I will tell you this is not for the faint of heart and neither was the class. Now more than ever it is an important part of anyone doing data science.
This is a straight ahead approach to Linear Algebra. Written in a straight ahead fashion addressing fundamentals of matrix theory like simultaneous equations, determinants and eigenvalue analysis. I probably have a high affinity for this book because it is a first edition published in 1965.
Programming and Code Development
Colloquially known as TAOCP, I am not going to say much about these tomes except you should at a minimum know what they are and have in them. Not for the faint of heart if you just want to do some PHP or ruby on rails. Having had the experience to take a class from Professor Knuth with variations of these via class notes being the backdrop, I cannot say enough about the text. For the true software engineer a must for the arsenal.
IMHO this is a great book for the thought processes that lead to creating production products. It was a seminal work in the area before all of the Agile books hit the market. While I myself am not a big proponent of ‘software process’ this book gets you in the modus operandi of how to think not just code.
I was having discussion the other day with some I respect in the audio, semantics and data science field and we noted that long before you could spin up a virtual machine in the proverbial cloud there was a field called real time signal processing - where you didnt have a database. All the processing of streams were in real time – in memory. Funny how the vissistudes of cyclialty come back to haunt you think Big Data. I highly recommend this text. Great examples of frequency and array based processing. When everyone was teaching time domain behavior Professor Lindquist was teaching the law of dual domains and easier math. It also covers various kernel transforms never covered in other text before the term “wavelet” came in to vogue.
So there is a first cut in a multi volume blog that I will continue. I trust this was helpful. If you find any of these text useful feel free to contact me with what you are working on or considering. Also if there are recommendations that you find particularly informative as far as a certain text goes drop me a line.