Lean Analytics and Lean UX – Two new guides to better business and user experiences – #bookreview

Okay, how are we leaning today? Leaning in? Leaning back? Leaning to the left or right? Leaning over? Or just leaning toward chucking all “hot new” postures that supposedly help us pose ourselves for career success?

Here’s some good news. None of the above leanings are topics in two new books from O’Reilly’s popular “Lean” series, edited by Eric Ries.

Lean Analytics deals with using data to help you determine if there is a profitable need for the product or service you hope to offer with a startup business. Lean UX, meanwhile, deals with the process of designing a better user experience (UX) for a company’s apps, website or other products.  Here are short reviews of each book:

Lean Analytics
Use Data to Build a Better Startup Faster
Alistair Croll and Benjamin Yoskovitz
(O’Reilly – hardback, Kindle)

“Entrepreneurs,” the authors state, “are particularly good at lying to themselves. Lying may even be a prerequisite for succeeding as an entrepreneur–after all, you need to convince others that something is true in the absence of good, hard evidence. You need believers to take a leap of faith with you. As an entrepreneur, you need to live in a semi-delusional state just to survive the inevitable rollercoaster ride of running your startup.”

But…you also need cold, hard data. And what you learn from that data may not mesh well with the lie you are living as you try to start a new business from scratch. Yet, it may save you from failing and wasting a lot of money.

“Your delusions,” the authors argue, “no matter how convincing, will wither under the harsh light of data. Analytics is the necessary counterweight to lying, the yin to the yang of hyperbole. Moreover, data-driven learning is the cornerstone of success in startups. It’s how you learn what’s working and iterate toward the right product and market before the money runs out.”

Lean Analytics builds on the Lean Startup process developed by Eric Ries. In today’s digital world, the authors explain, “[w]e’re in the midst of a fundamental shift in how companies are built. It’s vanishingly cheap to create the first version of something. Clouds are free. Social media is free. Competitive research is free. Even billing and transactions are free.”

Taken together, these facilities mean “you can build something, measure its effect, and learn from it to build something better next time. You can iterate quickly, deciding early on if you should double down on your idea or fold and move on to the next one.”

Their 409-page book is not quick reading. But it deserves attention and study, whether you want to start a business, already have started a business, or hope to revamp and improve a business that has been in operation for some time. Lean Analytics presents many examples and case studies that illustrate how you can gather and analyze existing data, then test products or services to determine if they are something that customers actually need, want and will use.

With new data from the tests and the ability to continue testing, you can modify your product or service and focus more resources, energy, and time on improving and refining what will work best for your customers–and your bottom line.

***

Lean UX
Applying Lean Principles to Improve User Experience
Jeff Gothelf with Josh Seiden
(O’Reilly - hardback, Kindle)

“Lean UX is a collaborative process,” the two authors of this book emphasize. “It brings designers and non-designers together in co-creation. It yields ideas that are bigger than those of the individual contributors. But it’s not design-by-committee. Instead, Lean UX increases a team’s ownership over the work by providing an opportunity for all opinions to be heard much earlier in the process.”

For example, forget the notion of a web designer hiding in an office for a week or so and then emerging with what he or she insists will be a “masterpiece” as the company’s new home page.

Particularly in software development, a key aspect of Lean and Agile development theories is the notion of creating a Minimum Viable Product (MVP). “Lean UX makes heavy use of the notion of MVP,” the two authors explain. “MVPs help test our assumptions–will this tactic achieve the desired outcome?–while minimizing the work we put into unproven ideas. The sooner we can find which features are worth investing in, the sooner we can focus our limited resources on the best solutions to our business problems. This concept is an important part of how Lean UX minimizes waste.”

The web designer’s “masterpiece” might work okay, but it also might offer costly confusions for customers and others visiting the website. Instead, Lean UX emphasizes collaboration, teamwork, testing prototypes, analyzing the results, gathering feedback from outsiders, revamping the project, testing it again–and continuing the process.

According to the writers, the most powerful tool in Lean UX is one that is basic to human beings: conversation. Indeed, conversation should be “the primary means of communication among team members.” Some of the other tools for collaboration also are basic: pencils, pens, notepads, whiteboards, blackboards, and simple paper templates that can spur discussions, opinions, and basic designs for the Minimum Viable Product and its successors, before moving the work to computers.

Lean UX is just 130 pages long. But it is rich with how-to examples, process descriptions, short case studies, clear steps, useful illustrations, and good examples that you can adapt and employ to create cheaper, faster, and better user experiences.

Si Dunn

Outsource It! — The good, bad, and ugly of offshoring tech projects – #bookreview

Outsource It!
A No-Holds-Barred Look at the Good, the Bad, and the Ugly of Offshoring Tech Projects
Nick Krym
(Pragmatic Bookshelf – paperback)

Like it or not, outsourcing tech projects is here to stay. It’s also there to stay, and everywhere else to stay.

There is no clear way that outsourcing will shrivel up and die within the interconnected and increasingly interdependent world economy.

So, perhaps it’s time to stop griping, resisting, and mouthing political slogans–and focus, instead, on finding ways to make the best of offshoring. There are ways to profit from its advantages. And there are ways to minimize the risks from its quirks, management challenges, traps and disadvantages.

Actually, some “offshoring” is “nearshoring.” To help keep development costs down, big corporations in North America sometimes farm out tech work to smaller companies and individual freelancers located in less-expensive areas of the United States, Canada, and Mexico.

“Inshoring” happens, too. U.S. firms move some of their overseas tech operations back to the States, and foreign companies establish some tech outsource operations in the United States, Canada, and Mexico. Their outsourcing is our insourcing.

Outsourcing veteran Nick Krym calls his new book Outsource It! “a down-to-earth guide to offshore outsourcing.” It is aimed, he says, at “technology professionals…working in small- to medium-sized companies or in the technology trenches of large organizations.”

Outsource It! is well-written and packed with good information and how-to steps, plus insights drawn from Krym’s experiences and the experiences of many others in real-world offshoring. His 25 years in the IT industry include 20 years working in offshore outsourcing.

If you work in outsource situations, or if you are helping manage or set up an outsource team, you can glean good information and how-to ideas from Krym’s pages. And, you likely will want to keep the book handy in your reference collection, because he covers many “soft skills that need to be reinforced continuously until they become second nature.”

The 244-page book is divided into five main parts:

1. Decide If, What, and How to Outsource
2. Find the Right Vendors
3. Negotiate Solid Contracts
4. Lead Distributed Engagements
5. Keep Risks Under Control

Three appendices take you inside the positives and negatives of outsourcing to India, China, Russia, Central and Eastern Europe, Ireland, Israel, South America, Central America, Mexico, Canada, and the rural United States.

Other appendices offer: an “Outsourcing Readiness Assessment Checklist”; a summary of “Vendor Search Criteria”; an “Outsourcing Checklist”; and an “Offshore Vendor Technical Assessment” process.

As someone who previously worked in multinational software development, on projects involving teams in the U.S., Canada, France, Italy, Sweden and China, I found myself particularly agreeing with Krym’s assessments of software outsourcing.

“Many companies think that QA—software testing—is a logical function to outsource,” he reports. He offers several reasons why this not always “the most prudent approach” and describes what it takes to make offshore QA work.

For example: “The first rule of setting up a productive offshore team,” he stresses,” is to use QA professionals rather than software developer rejects or English major graduates.”

It is likewise vital to find “a solid QA lead—someone who is sufficiently technical, understands the process and requirements, and can manage the team.”

Krym further emphasizes that “[t]he cost difference between local and outsourced QA engineers is not always as dramatic as it is for developers.”

And: “Poor QA management can generate huge amounts of useless work, producing hard-to-manage artifacts and creating unhealthy team dynamics.”

Nick Krym’s new book is an excellent guide to the ins, outs and complex gray areas of outsourcing technology projects. And it’s not just for managers and executives. Employees, freelancers, and leaders of start-ups also can find ways to benefit and profit from the knowledge and experience Outsource It! offers.

– Si Dunn

All for Search and Search for All: 3 New Books for Putting Search to Work – #bookreview

Seek and ye shall find.

That’s the theory behind the still-debated benefits of digging through Big Data to uncover new, overlooked, or forgotten paths to greater profits and greater understanding.

Big Data, however, is here to stay (and get bigger). And search is what we do to find and extract useful nuggets and diamonds and nickels and dimes of information.

O’Reilly Media recently has published three new, enlightening books focused on the processes, application, and management of search: Enterprise Search by Martin White, Mastering Search Analytics by Brent Chaters, and Search Patterns by Peter Morville and Jeffery Callender.

Here are short looks at each.

Enterprise Search
Martin White
(O’Reilly, paperback, Kindle)

Start with this book if you’re just beginning to explore what focused search efforts and search technology may be able to do for your company.

The book’s key goal is “to help business managers , and the IT teams supporting them, understand why effective enterprise-wide search is essential in any organization, and how to go about the process of meeting user requirements.”

You may think, So what’s the big deal? Just put somebody in a cubicle and pay them to use Google, Bing, and a few other search engines to find stuff.

Search involves much more than that. Even small businesses now have large quantities of potentially profitable information stored internally in documents, emails, spreadsheets and other formats. And large corporations are awash in data that can be mined for trends, warnings, new opportunities, new product or service ideas, and new market possibilities, to name just a few.

The goal of Enterprise Search is to help you set up a managed search environment that benefits your business but also enables employees to use search technology to help them do their jobs more efficiently and productively.

Yet, putting search technology within every worker’s reach is not the complete answer, author Martin White emphasizes.

“The reason for the well-documented lack of satisfaction with a search application,” he writes, “is that organizations invest in technology but not staff with the expertise and experience to gain the best possible return on the investment….”

Enterprise Search explains how to determine your firm’s search needs and how to create an effective search support team that can meet the needs of employees, management, and customers.

Curiously, White waits until his final chapter to list 12 “critical success factors” for getting the most from enterprise-wide search capabilities.

Perhaps, in a future edition, this important list will be positioned closer to the front of the book.

Mastering Search Analytics
Brent Chaters
(O’Reilly - paperback, Kindle)

This in-depth and well-illustrated guide details how a unified, focused search strategy can generate greater traffic for your website, increase conversion rates, and bring in more revenue.

Brent Chaters explains how to use search engine optimization (SEO) and paid search as part of an effective, comprehensive approach.

Key to Chaters’ strategy is the importance of bringing together the efforts and expertise of both the SEO specialists and the Search Engine Marketing (SEM) specialists — two groups that often battle each other for supremacy within corporate settings.

“A well-defined search program should utilize both SEO and SEM tactics to provide maximum coverage and exposure to the right person at the right time, to maximize your revenue,” Chaters contends. “I do not believe that SEO and SEM should be optimized from each other; in fact, there should be open sharing and examination of your overall search strategy.”

His book is aimed at three audiences: “the search specialist, the marketer, and the executive”–particularly executives who are in charge of search campaigns and search teams.

If you are a search specialist, the author expects that “you understand the basics of SEO, SEM, and site search (meaning you understand how to set up a paid search campaign, you understand that organic search cannot be bought, and you understand how your site search operates and works.)”

Search Patterns
Peter Morville and Jeffery Callender
(O’Reilly – paperback, Kindle)

“Search applications demand an obsessive attention to detail,” the two authors of this fine book point out. “Simple, fast, and relevant don’t come easy.”

Indeed, they add, “Search is not a solved problem,” but remains, instead, “a wicked problem of terrific consequence. As the choice of first resort for many users and tasks, search is the defining element of the user experience. It changes the way we find everything…it shapes how we learn and what we believe. It informs and influences our decisions and, and it flows into every noon and cranny….Search is among the biggest, baddest, most disruptive innovations around. It’s a source of entrepreneurial insight, competitive advantage, and impossible wealth.”

They emphasize: “Unfortunately, it’s also the source of endless frustration. Search is the worst usability problem on the Web….We find too many results or too few, and most regular folks don’t know where to search, or how….business goals are disrupted by failures in findability…[and] “Mobile search is a mess.”

Ouch!

Colorfully illustrated and well-written, Search Patterns is centered around major aspects in the design of user interfaces for search and discovery. It is aimed at “designers, information architects, students, entrepreneurs, and anyone who cares about the future of search.”

It covers the key bases, “from precision, recall, and relevance to autosuggestion and faceted navigation.” It looks at how search may be reshaped in the future. And, very importantly, it also joins the growing calls for collaboration across disciplines and “tearing down walls to make search better….”

– Si Dunn

Go APE (Author, Publisher, Entrepreneur) with Guy Kawasaki & Shawn Welch – #bookreview #amwriting

APE: Author, Publisher, Entrepreneur—How to Publish a Book
Guy Kawasaki and Shawn Welch
(Nononina Press, Kindle)

Okay, confession time. I know a bit about the book business—what used to be the book business.

Years ago, I was a freelance developmental book editor for a trio of well-known publishing houses; I’ve had a couple of book agents; books I wrote have been put into print by not-so-major publishers (and later dropped out of print); I’ve written hundreds of book reviews; and I’ve self-published a few books and ebooks: nonfiction, fiction and poetry.

To misquote the late actor-comedian W.C. Fields, on the whole, I’d rather be in self-publishing now.  There isn’t much of an alternative.

And not just basic self-publishing but artisanal self-publishing, which Guy Kawasaki and Shawn Welch define, in their well-written and well-designed new book, as “a new, cool form of publishing…authors lovingly crafting their books with total control over the process.”

Many writers, of course, already are trying to do that, often with abysmal results, because it’s not enough to commit a book to print (or its digital equivalent) and then wait for the world to recognize your genius and surge forward to buy it on Amazon.

To succeed in self-publishing, you really do have to be, as Kawasaki and Welch contend, an APE: an author, a publisher, and an entrepreneur. 

With APE, Kawasaki and Welch aim to “help people take control of their writing careers by publishing their books. The thesis of APE is simple but powerful: When a self-publisher successfully fills three roles—author, publisher and entrepreneur—the potential benefits are greater than with traditional publishing.”

There’s plenty of truth in that. Three publishers turned down my Vietnam War memoir Dark Signals, even after it received a prestigious award. And several other publishers did not bother to respond to my queries. So I published it myself as a CreateSpace paperback and Kindle ebook, both available through Amazon.

It has not been a runaway best-seller; I knew from the outset that I was writing for a limited audience: readers of military memoirs. Yet several hundred copies have been ordered thus far. And a book that I really needed to push out of my soul finally is out there for posterity, with five-star reviews.

No doubt I could have sold more copies at the outset if I had had APE in hand. Knowing the traditional book business is one thing. Knowing the new ways of book creation and marketing are quite another.   

Filling the three roles — author, publisher and entrepreneur — is “challenging, but they are not impossible—especially if people who have done it before explain it to you.” That’s the key premise behind APE. Kawasaki, a successful author, has become a successful self-publisher with help from Shawn Welch, and together, they are now offering up their hard-earned secrets in a 300-page book that many authors will want to read, repeatedly.

Indeed, many of us likely will value APE as a Chicago Manual of Style for self-publishing that also has entertaining writing and dozens of how-to tips thrown in for added value. APE is comprehensive. And it’s very realistic about what it takes to succeed as a self-published author.

Three points in particular stand out for me.

1. Yes, I have been a professional editor and proofreader of books. But I still should never do the final edits and proofreading of my own text. (Neither should you.) “The self-edited author is as foolish as the self-medicated patient,” Guy Kawasaki points out. Indeed, I have had to create new editions of at least two of my ebooks, because I found glaring typos that I had completely overlooked while doing my “final” edits. As Kawasaki notes: “The going rate for copyediting is $35 per hour, and copyeditors can work their magic at the rate of roughly ten pages per hour (although this can vary depending on the complexity of the material), so you’ll pay approximately $1,000–$1,500 for a three-hundred-page manuscript. This is one of the dumbest places to try to save money, because poor copyediting destroys the quality of your book.” (Unfortunately, you will have to sell a lot of ebooks to cover that cost.)
2. At least two of my CreateSpace books have boring covers. I am not a graphic artist, and I should not attempt to save money in the future by “designing” my own book covers or settling for one of the available “standard” covers. As Kawasaki notes: “Not to get too metaphysical, but a cover is a window into the soul of your book. In one quick glance, it needs to tell the story of your book and attract people to want to read it. Unless you’re a professional, hire a professional to create a great cover because, in spite of how the old saying goes, you can judge a book by its cover. Or at very least, people will judge a book by its cover.”
3. While I have dabbled at business for many years, I am not much of an entrepreneur. And I don’t have the soul of a self-promoting guerilla marketer. I grew up believing modesty is a virtue. (Or, perhaps I merely had that notion spanked into my britches when I was an Eisenhower-era kid.) In any case, when my first books were published, others hired by the publishers did the editing, bragging, selling and distribution. Sometimes I talked to small groups of people and signed a few autographs. But mostly, I just stayed home, started a new project, and waited for the (small) checks to arrive. Now, in APE’s chapter on “How to Build an Enchanting Personal Brand,” Kawasaki states: “Call me idealistic, but your platform is only as good as your reality. If you suck as a person, your platform will suck too.” Cool. Memo to self: Improve personal enchantment platform immediately. (By the way, Guy and Shawn, I would add a comma between “suck” and “too.” You’re welcome.) Seriously, if we self-publish books, we have to sell ourselves to readers, right along with, and often ahead of, our books. And the eight chapters of APE’s “Entrepreneur” section provide excellent guidelines on how to do that.

Even if you already know a lot about self-publishing and self-marketing books, if you’ll go APE, you can learn some profitable new tricks from Guy Kawasaki and Shawn Welch.

– Si Dunn

QuickBooks 2013: The Missing Manual – Updated and now it’s ‘Official’ – #bookreview

QuickBooks 2013: The Missing Manual
By Bonnie Biafore
(O’Reilly, paperback - Kindle)

Bonnie Biafore’s latest version of her popular “Missing Manual” QuickBooks how-to guide has gained a significant endorsement. It’s now labeled “The Official Intuit Guide” to QuickBooks 2013.

Once again, the updated guidebook provides clear, well-illustrated, step-by-step instructions on how to use the Windows edition of QuickBooks 2013, the most popular version of Intuit’s product, particularly in small businesses.

The 736-page book also gives some basic how-to information and advice on bookkeeping and  accounting – enough to get you past some confusing stumbling blocks as you set up a business and its accounts, but not enough that you should try to skip legal advice and focused training.

“QuickBooks isn’t hard to learn,” the author emphasizes. “Many of the features that you’re familiar with from other programs work just the same way in QuickBooks—windows, dialog boxes, drop-down lists, and keyboard shortcuts, to name a few. And with each new version, Intuit has added enhancements and new features to make your workflow smoother and faster. The challenge is knowing what to do according to accounting rules, and how to do it in QuickBooks.”

The 2013 edition of QuickBooks, she points out, “sports a brand-new look that simplifies the interface, removes clutter, and presents features and options in a much more organized and consistent way.” Her book also describes several features that are new in the 2013 release.

Three words of caution: First, this book does not cover non-USA versions of QuickBooks 2013. Second, the Pro and Premier editions of the software cannot handle more than 14,500 unique inventory items or 14,500 contact names.  And third, the author points out, “QuickBooks for Mac differs significantly from the Windows version, so this book isn’t meant to be a guide to the Mac version of the program.”

There are many different versions of the QuickBooks 2013 software. But this “Missing Manual” puts most of its focus on QuickBooks 2013 Pro, ”because its balance of features and price make it the most popular edition.” However, the author also provides “notes about features offered in the Premier edition, which is one step up from Pro.”

For many of us in small business, a QuickBooks “Missing Manual”  is  the best QuickBooks how-to guide, period. And Intuit’s new “Official Guide” stamp of approval should help seal the deal for others seeking the best QuickBooks reference.

The new edition of Bonnie Biafore’s book is divided into five parts containing a total of 26 chapters and two appendices.

Part One covers “Getting Started.” It starts with “Creating a Company File” and “Getting Around in QuickBooks” and advances to setting up accounts, customers, jobs, vendors, items, lists, and managing QuickBooks files.

Part Two’s focus is “Bookkeeping,” and its chapters covers everything from tracking time and mileage to paying for expenses, invoicing, managing accounts receivable, generating financial statements and performing end-of-year tasks.

“Managing Your Business” is the focus of Part Three. These chapters cover managing inventory, budgeting and planning, and working with reports.

“QuickBooks Power” is the title of Part Four. It covers using QuickBooks with online banking services, configuring preferences in QuickBooks to fit your company, integrating QuickBooks with other programs, customizing QuickBooks, and keeping your QuickBooks data secure.

Part Five contains two appendices: “Installing QuickBooks” and “Help, Support, and Other Resources.”

As usual, this “Missing Manual” does not contain a CD, but it does provide a link where “every single Web address, practice file, and piece of downloadable software mentioned In this book is available….”

QuickBooks 2013: The Missing Manual can help you get a small business set up and off the ground while you are learning the program.  But if you don’t have a reasonably good background in bookkeeping and accounting, do not try to rely on the software alone to save you. Get competent help from a consultant and legal advice from someone besides your rich Uncle Bob. And get some training any way you can, as soon as you can.

Then, once you can afford it, hire good people to help you with  your bookkeeping and accounting, while you focus on the bigger picture, growing your business with the help of QuickBooks 2013’s budgeting, planning, forecast, report, contact synchronization, lead tracking, and to-do list features.

One other caution: QuickBooks has a specialized edition specifically for nonprofit organizations. It is more expensive than the Pro package. So some people try to save money and use the Pro package to manage a small nonprofit. But there can be confusions involving some of the terminology, transactions and reports. And there may not be workarounds for some situations. You may be better off buying the QuickBooks Nonprofit edition.

By the way, QuickBooks 2013: The Missing Manual can be used to learn features in earlier versions of QuickBooks. Of course, doing so and seeing what’s missing may convince you to upgrade. And the 2013 screens will appear somewhat different from what you see. If you have an older version of QuickBooks and don’t plan to upgrade soon, consider looking online for one of the earlier editions of this book.

– Si Dunn

Big Data Book Blast: Hadoop, Hive…and Python??? – #programming #bookreview

Big Data is hothotHOT. And O’Reilly recently has added three new books of potential interest to Big Data workers, as well as those hoping to join their ranks.

Hadoop, Hive and–surprise!—Python are just a few of the hot tools you may encounter in the rapidly expanding sea of data now being gathered, explored, stored, and manipulated by companies, organizations, institutions, governments, and individuals around the planet. Here are the books:

Hadoop Operations
Eric Sammer
(O’Reilly, paperback – Kindle)

“Companies are storing more data from more sources in more formats than ever before,” writes Eric Sammer, a Hadoop expert who is principal solution architect at Cloudera. But gathering and stockpiling data is only “one half of the equation,” he adds. “Processing that data to produce information is fundamental to the daily operations of every modern business.”

Enter Apache Hadoop, a “pragmatic, cost-effective, scalable infrastructure” that increasingly is being used to develop Big Data applications for storing and processing information.

“Made up of a distributed filesystem called the Hadoop Distributed Filesystem (HDFS) and a computation layer that implements a processing paradigm called MapReduce, Hadoop is an open source, batch data processing system for enormous amounts of data. We live in a flawed world, and Hadoop is designed to survive in it by not only tolerating hardware and software failures, but also treating them as first-class conditions that happen regularly.”

Sammer adds: “Hadoop uses a cluster of plain old commodity servers with no specialized hardware or network infrastructure to form a single, logical, storage and compute platform, or cluster, that can be shared by multiple individuals or groups. Computation in Hadoop MapReduce is performed in parallel, automatically, with a simple abstraction for developers that obviates complex synchronization and network programming. Unlike many other distributed data processing systems, Hadoop runs the user-provided processing logic on the machine where the data lives rather than dragging the data across the network; a huge win for performance.”

Sammer’s new, 282-page book is well written and focuses on running Hadoop in production, including planning its use, installing it, configuring the system and providing ongoing maintenance. He also shows “what works, as demonstrated in crucial deployments.”

If you’re new to Hadoop or still getting a handle on it, you need Hadoop Operations. And even if you’re now an “old” hand at Hadoop, you likely can learn new things from this book. “It’s an extremely exciting time to get into Apache Hadoop,” Sammer states.

Programming Hive
Eric Capriolo, Dean Wampler, and Jason Rutherglen
(O’Reilly, paperback – Kindle)

“Hive,” the three authors point out, “provides an SQL dialect, called Hive Query Language (abbreviated HiveQL or just HQL), for querying data stored in a Hadoop cluster.”

They add: “Hive is most suited for data warehouse applications, where relatively static data is analyzed, fast response times are not required, and when data is not changing rapidly.”

Their well-structured and well-written book shows how to install and test Hadoop and Hive on a personal workstation – “a convenient way to learn and experiment with Hadoop.” Then it shows “how to configure Hive for use on Hadoop clusters.”

They also provide a brief overview of Hadoop and MapReduce before diving into Hive’s command-line interface (CLI) and introductory aspects such as how to embed lines of comments in Hive v0.80 and later.

From there, the book flows smoothly into HiveQL and how to use its SQL dialect to query, summarize, and analyze large datasets that Hadoop has stored in its distributed filesystem.

User documentation for Hive and Hadoop has been sparse, so Programming Hive definitely fills a solid need. Significantly, the final chapter presents several “Case Study Examples from the User Trenches” where real companies explain how they have used Hive to solve some very challenging problems involving Big Data.

Python for Data Analysis
Wes McKinney
(O’Reilly, paperback – Kindle)

No, Python is not the first language many people think of when picturing large data analysis projects. For one thing, it’s an interpreted language, so Python code runs a lot slower than code written in compiled programming languages such as C++ or Java.

Also, the author concedes, “Python is not an ideal language for highly concurrent, multithreaded applications, particularly applications with many CPU-bound threads.” The software’s global interpreter lock (GIL) “prevents the interpreter from executing more than one Python bytecode instruction at a time.”

Thus, Python will not soon be challenging Hadoop to a Big Data petabyte speed duel.

On the other hand, Python is reasonably easy to learn, and it has strong and widespread support within the scientific and academic communities, where a lot of data must get crunched at a reasonable clip, if not at blinding speed.

And Wes McKinney is the main author of pandas, Python’s increasingly popular open source library for data analysis. It (pandas) is “designed to make working with structured data fast, easy, and expressive.”

His book makes a good case for using Python in at least some Big Data situations. “In recent years,” he states, “Python’s improved library support (primarily pandas) has made it a strong alternative for data manipulation tasks. Combined with Python’s strength in general purpose programming, it is an excellent choice as a single language for building data-centric applications.”

Much of this well-written, well-illustrated book “focuses on high-performance array-based computing tools for working with large data sets.” It uses a case-study-examples approach to demonstrate how to tackle a wide range of data analysis problems, using Python libraries that include pandas, NumPy, matplotlib, and IPython, “the component in the standard scientific Python toolset that ties everything together.”

By the way, if you have never programmed in Python, check out the end of McKinney’s book. An appendix titled “Python Language Essentials” gives a good overview of the language, with a specific bias toward “processing and manipulating structured and unstructured data.”

If you do scientific, academic, or business computing and need to crunch and visualize a lot of data, definitely check out Python for Data Analysis.

You may be pleasantly surprised at how well and how easily Python and its data-analysis libraries can do the job.

– Si Dunn

Enterprise Games – How to build a better 21st-century business with game mechanics – #business #bookreview

Enterprise Games: Using Game Mechanics to Build a Better Business
Michael Hugos
(O’Reilly, paperback – Kindle)

Can 21^st-century games and gamers attack and destroy the top-down, assembly-line thinking that still keeps many businesses firmly rooted in the previous century?

 Michael Hugos’ compelling new book makes a solid case that they can. Game mechanics, he argues, can reshape how workers work, how organizations are managed, and how business goals get accomplished in today’s volatile global economy.

“Games and the associated technology we currently refer to as video games offer us more than just a diversion and escape from difficult times,” contends Hugos. “They offer us field-tested models to use for organizing companies and performing complex and creative tasks. They offer clear and compelling examples for how people can work together, build their careers, and earn a living in rapidly changing and unpredictable environments.”

Hugos, principal at the Center for Systems Innovation, offers his well-written views in a 199-page book “loosely divided into three parts.”

Part One focuses on “ideas and case studies to illustrate how games can provide operating models to follow for redesigning work.”

Part Two presents “a discussion of games and game mechanics that are relevant to the way work is done.” He includes “specific examples, pictures, and case studies to show how game techniques and technologies can be applied to the design of new business systems and workflows.”

Part Three “describes business and social impacts of combining technology from video games with in-house corporate systems, consumer technology, and cloud computing. The book concludes with a discussion about where this is all going and what it might mean for the future of work.”

During the coming months, Enterprise Games may spur many discussions and arguments at all levels of enterprise. And these may lead to some business-model reorganizations not only in Corporate America but elsewhere in the interconnected global economy.

For these changes to happen, however, many company leaders will have to stop thinking “top down” and learn to adapt ”the four traits of a game…goal, rules, feedback system, and voluntary participation” to how they to structure and operate a business.

“We all have a sense of what a game is,” Hugos notes. But most of us also have been taught that “play” is not “work.” Enterprise Games shows how the two concepts can be brought together in ways that can make companies more competitive and more profitable in these uncertain times.

– Si Dunn

Ethics of Big Data – Thoughtful insights into key issues confronting big-data ‘gold mines’ – #management #bookreview

Ethics of Big Data
Kord Davis, with Doug Patterson
(O’Reilly, paperback – Kindle)

“Big Data” and how to mine it for profit are red-hot topics in today’s business world. Many corporations now find themselves sitting atop virtual gold mines of customer information. And even small businesses now are attempting to find new ways to profit from their stashes of sales, marketing, and research data. 

Like it or not, you can’t block all of the cookies or tracking companies or sites that are following you, and each time you surf the web, you leave behind a “data exhaust” trail that has monetary value to others. Indeed, one recent start-up, Enliken, (“Data to the People”), is offering a way for computer users to gain some control over their data exhaust trail’s monetary value and choose who benefits from it, including some charities.

Ethics of Big Data does not seek to lay down a “hard-and-fast list of rules for the ethical handling of data.” The new book also doesn’t “tell you what to do with your data.” Its goals are “to help you engage in productive ethical discussions raised by today’s big-data-driven enterprises, propose a framework for thinking and talking about these issues, and introduce a methodology for aligning actions with values within an organization.”

It’s heady stuff, packed into just 64 pages. But the book is well written and definitely thought-provoking. It can serve as a focused guide for corporate leaders and others now hoping to get a grip on their own big-data situations, in ways that will not alienate their customers, partners, and stakeholders.

In the view of the authors: “For both individuals and organizations, four common elements define what can be considered a framework for big data:

• “Identity – What is the relationship between our offline identity and our online identity?”
• “Privacy – Who should control access to data?”
• “Ownership – Who owns data, can rights be transferred, and what are the obligations of people who generate and use that data?”
• “Reputation – How can we determine what data is trustworthy? Whether about ourselves, others, or anything else, big data exponentially increases the amount of information and ways we can interact with it. This phenomenon increases the complexity of managing how we are perceived and judged.”

Big-data technology itself is “ethnically neutral,” the authors contend, and it “has no value framework. Individuals and corporations, however, do have value systems, and it is only by asking and seeking answers to ethical questions that we can ensure big data is used in a way that aligns with those values.”

At the same time: “Big data is pushing corporate action further and more fully into individual lives through the sheer volume, variety, and velocity of the data being generated. Big-data product design, development, sales, and management actions expand their influence and impact over individuals’ lives that may be changing the common meanings of words like privacy, reputation, ownership, and identity.”

What will happen next as (1) big data continues to expand and intrude and (2) people and organizations  push back harder, is still anybody’s guess. But matters of ethics likely will remain at the center of the conflicts.

Indeed, some big-data gold mines could suffer devastating financial and legal cave-ins if greed is allowed to trump ethics.

– Si Dunn

The Connected Company – Restructure now or die in today’s hyperconnected economy – #bookreview

The Connected Company
Dave Gray, with Thomas Vander Wal
(O’Reilly, hardback – Kindle)

If you buy only one business management book this year, make it this one. It’s that good, and definitely timely.

Whether your organization chart stretches across continents or consists of just you, your smart phone and your computer, you can learn important insights and paths for new action from Dave Gray’s and Thomas Vander Wal’s well-written book.

“Competitive intensity is rising all over the world,” they emphasize. “Global competition and the Web have given customers more choices than they have ever had before. This means that customers can choose from an ever-widening set of choices, and it seems that variety only breeds more variety. The more choices that become available, the more choices it seems that people want.”

At the same time, like it or not: “The balance of power is shifting from companies to the networks that surround them. Connected, communicating customers and employees have more choices, and more amplified voices, than ever before. They have more knowledge than ever before. These trends are only increasing with time. This means the network—customers, partners, and employees—will increasingly set the agenda, determine the parameters, and make the decisions about how they interact with companies.”

And: “By changing the way we create, access, and share information, social networks are changing the power structure in society.”

Today, one negative tweet, blog post, or video that goes viral can wreak havoc within a company (or political campaign), disrupt careers, damage or destroy expensive advertising campaigns, and turn potential and existing customers away in droves.

In an economy increasingly service-driven, your factory-model training and mentality is now completely obsolete. You must be connected, you must stay engaged with customers and the rest of the world, and you must be able to respond to rumors and actual bad news as quickly and completely as you respond to orders from your best customers.

The Connected Company is organized into five parts that clearly spell out the problems and the achievable solutions.

• Part One: Why change? – “Customers are adopting disruptive technologies faster than companies can adapt.” And: “Customers are connecting, forming networked communities that allow them to rapidly share information and self-organize into powerful interest groups.” To survive, you have to be more responsive to what they need and increasingly have the power to demand.
• Part Two: What is a connected company? – “To adapt companies must operate not as machines but as learning organisms, purposefully interacting with their environment and continuously improving, based on experiments and feedback.”
  Part Three: How does a connected company work? – “A connected company learns and adapts by distributing control to the points of interaction with customers, where semi-autonomous pods pursue a common purpose supported by platforms that help them organize and coordinate their activities.”
• Part Four: How do you lead a connected company? – “Connected companies are living, learning networks that live within larger networks. Power in networks comes from awareness and influence, not control. Leaders must create an environment of clarity, trust, and shared purpose, while management focuses on designing and tuning the system that supports learning and performance.”
• Part Five: How do you get there from here? – “Connected companies today are the exception, not the rule. But as long as the environment is characterized by change and uncertainty, connected companies will have the advantage. There are four ways your company can start that journey today….”

The traditional hierarchy model of business structure still works when your markets remain stable. But when is the last time, lately, that that actually has happened? Companies divided into functions increasingly go awry in times of uncertainty, because those individual departments cannot adapt, change, and respond quickly enough. In the world of The Connected Company, “companies must organize differently. They must reorganize from hierarchies into holarchies, where every part can function as a whole unto itself.”

Gray and Vander Wal stress: “A connected company is flexible and resilient, able to adapt quickly to change. The path from divided to connected company is not simple or easy. But in an increasingly volatile world, it is also not optional.”

Fortunately, their book lays out some clear strategies and procedures, as well as imperatives,  for getting there.

– Si Dunn

The Data Journalism Handbook – Get new skills for a new career that’s actually in demand – #bookreview

The Data Journalism Handbook: How Journalists Can Use Data to Improve the News
Edited by Jonathan Gray, Liliana Bounegru, and Lucy Chambers
(O’Reilly, paperback – Kindle)

Arise, ye downtrodden, unemployed newspaper and magazine writers and editors yearning to be working again as journalists. Data journalism apparently is hiring.

Data journalism? I didn’t know, either, until I read this intriguing and hopeful collection of essays, how-to reports, and case studies written by journalists now working as, or helping train, data journalists in the United States and other parts of the world.

Data journalism, according to Paul Bradshaw of Birmingham City University, combines “the traditional ‘nose for news’ and ability to tell a compelling story with the sheer scale and range of digital information now available.”

Traditional journalists should view that swelling tide of information not as a mind-numbing, overwhelming flood but ”as an opportunity,” says Mirko Lorenz of Deutsche Welle. “By using data, the job of journalists shifts its main focus from being the first ones to report to being the ones telling us what a certain development actually means.”

He adds: “Data journalists or data scientists… are already a sought-after group of employees, not only in the media. Companies and institutions around the world are looking for ‘sense makers’ and professionals who know how to dig through data and transform it into something tangible.”

So, how do you transform yourself from an ex-investigative reporter now working at a shoe store into a prizewinning data journalist?

A bit of training. And, a willingness to bend your stubborn brain in a few new directions, according to this excellent and eye-opening book.

Yes, you may still be able to use the inverted-pyramid writing style and the “five W’s and H” you learned in J-school. But more importantly, you will now need to show you have some good skills in (drum roll, please)…Microsoft Excel.

That’s it? No, not quite.

Google Docs, SQL, Python, Django, R, Ruby, Ruby on Rails, screen scrapers, graphics packages – these are just a few more of the working data journalists’ favorite things. Skills in some these, plus a journalism background, can help you become part of a team that finds, analyzes and presents information in a clear and graphical way.

 You may dig up and present accurate data that reveals, for example, how tax dollars are being wasted by a certain school official, or how crime has increased in a particular neighborhood, or how extended drought is causing high unemployment among those who rely on lakes or rivers for income.

You might burrow deep into publically accessible data and come up with a story that changes the course of a major election or alters national discourse.

Who are today’s leading practitioners of data journalism? The New York Times, the Texas Tribune, the Chicago Tribune, the BBC, Zeit Online, and numerous others are cited in this book.

The Data Journalism Handbook grew out of MozFest 2011 and is a project of the European Journalism Centre and the Open Knowledge Foundation.

This book can show you “how data can be either the course of data journalism or a tool with which the story is told—or both.”

If you are looking for new ways to use journalism skills that you thought were outmoded, The Data Journalism Handbook can give you both hope and a clear roadmap toward a possible new career.

– Si Dunn