Scaling Big Data with Hadoop and Solr – A new how-to guide – #bigdata #java #bookreview

Scaling Big Data with Hadoop and Solr

Learn new ways to build efficient, high performance enterprise search repositories for Big Data using Hadoop and Solr
Hrishikesh Karambelkar
(Packt – paperback, Kindle)

This well-presented, step-by-step guide shows how to use Apache Hadoop and Apache Solr to work with Big Data.  Author and software architect Hrishikesh Karambelkar does a good job of explaining Hadoop and Solr, and he illustrates how they can work together to tackle Big Data enterprise search projects.

“Google faced the problem of storing and processing big data, and they came up with the MapReduce approach, which is basically a divide-and-conquer strategy for distributed data processing,” Karambelkar notes. “MapReduce is widely accepted by many organizations to run their Big Data computations. Apache Hadoop is the most popular open source Apache licensed implementation of MapReduce….Apache Hadoop enables distributed processing of large datasets across a commodity of clustered servers. It is designed to scale up from single server to thousands of commodity hardware machines, each offering partial computational units and data storage.”

Meanwhile, Karambelkar adds, “Apache Solr is an open source enterprise search application which provides user abilities to search structured as well as unstructured data across the organization.”

His book (128 pages in print format) is structured with five chapters and three appendices:

  • Chapter 1: Processing Big Data Using Hadoop MapReduce
  • Chapter 2: Understanding Solr
  • Chapter 3: Making Big Data Work for Hadoop and Solr
  • Chapter 4: Using Big Data to Build Your Large Indexing
  • Chapter 5: Improving Performance of Search while Scaling with Big Data
  • Appendix A: Use Cases for Big Data Search
  • Appendix B: Creating Enterprise Search Using Apache Solr
  • Appendix C: Sample MapReduce Programs to Build the Solr Indexes

Where the book falls short (and I have noted this about many works by computer-book publishers) is that the author simply assumes everything will go well during the process of downloading and setting up the software–and gives almost no troubleshooting hints. This can happen with books written by software experts that are also are reviewed by software experts. Their systems likely are already optimized and may not throw the error messages that less-experienced users may encounter.

For example, the author states: “Installing Hadoop is a straightforward job with a default setup….” Unfortunately, there are many “flavors” and configurations of Linux running in the world. And Google searches can turn up a variety of problems others have encountered when installing, configuring and running Hadoop.  Getting Solr installed and running likewise is not a simple process for everyone.

If you are ready to plunge in and start dealing with Big Data, Scaling Big Data with Hadoop and Solr definitely can give you some well-focused and important information.  But heed the “Who this book is for” statement on page 2: “This book is primarily aimed at Java programmers, who wish to extend Hadoop platform to make it run as an enterprise search without prior knowledge of Apache Hadoop and Solr.”

And don’t surprised if you have to seek additional how-to details and troubleshooting information from websites and other books, as well as from co-workers and friends who may know Linux, Java and NoSQL databases better than you do (whether you want to admit it or not).

Si Dunn

Making Java Groovy – An excellent how-to guide for Java developers – #programming #bookreview

Making Java Groovy

Kenneth A. Kousen
(Manning, paperback)

I have looked at a lot of programming how-to texts in recent years, and this is one of the best I have encountered. Kenneth A. Kousen does not try to get readers to abandon aging and bloated Java in favor of a language that’s sleeker, shinier and newer. Instead, he seeks to show us how to use Groovy’s features and capabilities to help simplify Java, to make working with it easier, more efficient and–yes, even a bit more enjoyable. Indeed, Kousen proposes that we should gradually blend more and more Groovy into our Java code. Hence the title, Making Java Groovy.

(By the way, I lived through the hippie Sixties and remember how weary many of us grew of endlessly hearing the happy word “groovy”–as in “It’s groovy, man, groovy! Everything’s groovy!” I can say honestly that I resisted looking at Groovy, the programming language, for a long time, primarily because of that long-ago, wince-inspiring memory.)

Groovy is an object-oriented programming language that works on the Java platform, and it is intended to complement Java, not replace it. As Guillaume Laforge, the Groovy program manager, explains in his foreword to Kousen’s book: “The idea [behind Groovy] was to empower users to be more productive by removing the boilerplate of Java, and to simplify their programming lives by giving them compelling and straightforward APIs [application programming interfaces] to work with.”

Ken Kousen emphasizes that, among other things, Groovy “adds new capabilities to existing Java classes.” It uses Java libraries. And it makes it easier to work with XML and JSON. Kousen adds that “the Groovy version of a Java class is almost always simpler and cleaner. Groovy is far less verbose and generally easier to read.”

His book has ten chapters and two appendices and is structured into three parts. The first part, titled “Up to Speed with Groovy,” focuses on some long-time Java problems that Groovy addresses and simplifies, and it presents some ways you can use Groovy’s features in Java. The obligatory “Hello, World!” example is presented, but Kousen’s code accesses Google Chart Tools and generates a cool (okay, a groovy) “Hello, World!” 3D pie chart, to which you can add new slices and labels. (But, yes, you do first get the bland and obligatory Groovy shell command-line groovy:000>println ‘Hello, World!’ to verify that your Groovy installation works. Then you get to create the pie chart.)

Kousen is an independent consultant and technical trainer who specializes “in all areas related to Java.” He presents several Java code examples and shows how much shorter they can be made by integrating some Groovy into the mix. “Java historically hasn’t played well with others [other programming languages],” he cautions. But he demonstrates how Java and Groovy can be integrated smoothly to solve some nagging Java irritations and shortcomings.

The book’s second section highlights “Groovy Tools” and emphasizes build processes and testing, “two of the major ways Groovy is often introduced into an organization….”

The third section, “Groovy in the Real World,” describes “the sorts of challenges Java developers face on a regular basis” and how Groovy can help overcome them. The author starts with the Spring framework, “probably the most commonly used open source project in the Java world” and illustrates how “Spring and Groovy are old friends and work together beautifully.”  He also examines: (1) how Groovy interacts with persistent storage; (2) REST-ful web services, “with an emphasis on the JAX-RS 2.0 specification”; and (3) using Groovy in web application development.

The book’s instructions for downloading and installing Groovy are positioned near the back in Appendix A. and seem a bit sparse. But this is okay, since Making Java Groovy is not intended for programming beginners. And I can confirm that writers really don’t like to have to pause a well-written introduction long enough to explain how to download and install the right software for several operating systems.

Following Kousen’s instructions, I installed Groovy fairly easily on a Windows 7 PC, using a Windows EXE installer. He also mentions (too briefly) the Eclipse Marketplace. Since I have Eclipse (with Java and Scala) installed on a Linux machine, I used the Marketplace to get the Groovy and Grails plug-in, too. (My wife did not want Groovy anywhere on her precious Mac, so I did not test that installation.)

“Groovy generates Java bytecodes that are interpreted by a Java virtual machine,” Kousen states. “This means you have to have Java installed to install Groovy.”  Note also: “You need a full Java Development Kit (JDK) rather than a Java Runtime Environment (JRE).” And the Standard Edition (SE) is fine.

Appendix B, “Groovy by Feature,” expands upon Chapter 2’s “Groovy by Example,” and this one seems to be oddly placed. “While some people learn best through short, simple code examples illustrating each concept,” Kousen explains, “others prefer to see basic concepts combined to solve actual problems.” Appendix B “walks through most of the major features of Groovy and provides short snippets of code illustrating them.”

I would be happier if Appendix B had been positioned as Chapter 3, instead. Many people can learn both by feature and by example. And there should be nothing wrong with using one approach to reinforce the other.

But this is minor nitpicking. Making Java Groovy is an excellent how-to guide that I predict will go a long way to help popularize Groovy programming–even among us old guys who still shudder at the verbal excesses of our youth, when we proclaimed “everything” was “groovy.”

Si Dunn

OpenGL ES 2 for Android – A fine quick-start guide for new developers – #android #programming #bookreview

OpenGL ES 2 for Android

A Quick-Start Guide
Kevin Brothaler
(Pragmatic Bookshelf – paperback)

Yes, the timing might seem a bit strange, releasing an OpenGL ES 2 book in early July, 2013, barely a month before the August release of OpenGL ES 3.

However, OpenGL ES 3 is backward-compatible with OpenGL ES 2. And the steps and techniques you can learn in this Open GL ES 2 book for Android are forward-compatible to OpenGL ES 3. Many also are applicable to iOS WebGL or HTML5 WebGL.

This “quick-start guide” assumes you have some experience with Java and Android, and it quickly jumps into creating OpenGL applications for Android. You install software tools such as the Java Development Kit (JDK) and the Android Software Development Kit and create a simple test project. Then you dive into developing and enhancing a 3D game project —  “a simple game of air hockey” — for the remainder of the book.

OpenGL ES 2 for Android is nicely illustrated, well-written, and cleanly organized with short paragraphs and short code examples that clearly have been tested. It is a fine quick-start guide, particularly for developers looking into OpenGL for the first time.

Some math skills are required to develop the air hockey game. But the author does a nice job of explaining and illustrating the math examples, as well.

Kevin Brothaler has extensive experience in Android development. He founded Digipom, a mobile software development shop, and he manages an online set of OpenGL tutorials for Android and WebGL: Learn OpenGL ES.

Si Dunn

Programming Groovy 2 – For the experienced Java developer seeking dynamic productivity – #programming #bookreview

Programming Groovy 2

Dynamic Productivity for the Java Developer

Venkat Subramaniam
(Pragmatic Bookshelf, paperback)


The programming language
Groovy has a bit of a checkered past. Before it reached Release 1 in early 2007, it was almost abandoned because of a series of development problems. But some dedicated developers reworked it, gave it new life, and helped it gain acceptance in a widening array of commercial projects. Release 2.0 became available last year, and you can download 2.1, with 2.2 in beta. Groovy is doing groovy now, thank you very much (and thank the Grails web application framework, too).

In his new book, which keys off of his 2008 Groovy 1 edition, Dr. Venkat Subramaniam  describes Groovy 2 as “lightweight, low-ceremony, dynamic, object-oriented, and runs on the JVM [the Java Virtual Machine].”

He notes: “Groovy is open sourced under the Apache License, version 2.0. It derives strength from various languages such as Smalltalk, Python, and Ruby, while retaining a syntax familiar to Java programmers. Groovy compiles into Java bytecode and extends the Java API and libraries. It runs on Java 1.5 and newer. For deployment, all we need is a Groovy Java archive (JAR) in addition to the regular Java stuff, and we’re all set.”

Groovy is not for coding beginners, nor is it a means to avoid learning Java. This book–well written and nicely illustrated with short code examples and screenshots–”is aimed at Java programmers who already know the JDK [Java Development Kit] well but are interested in learning the Groovy language and its dynamic capabilities,” Dr. Subramaniam says.

He has organized his 19-chapter, 333-page book into three major parts:

Part I: Beginning Groovy – Focuses on the fundamentals of the language but deliberately skips the basics of programming. The book, after all, is aimed at “experienced Java programmers.”

Part II: Using Groovy – Shows “how to use Groovy for everyday coding–working with XML, accessing databases, and working with multiple Java/Groovy classes and scripts….” Also delves into Groovy extensions and additions to the JDK.

Part III: “MOPping Groovy” – The odd title may conjure up a brief image of mopping up spilled gravy. But this part deals with “Groovy’s metaprogramming capabilities….” The coverage includes: (1) metaobject protocol (MOP); (2) “how to do AOP-like operations in Groovy” [AOP = “aspect-oriented programming”]; and (3) “dynamic method/property discovery and dispatching,” as well as Groovy’s “compile-time metaprogramming capability….”

If you are a Java developer seeking new tools. new challenges, and new horizons, this could be the right time and right way to get your groove on: with Groovy 2 and Dr. Venkat Subramaniam’s fine how-to guide.

“Groovy is an attractive language for a number of reasons,” the author says, naming four key ones:

“It has a flat learning curve.”

“It follows Java semantics.”

“It bestows dynamic love.”

“It extends the JDK.”

“Groovy,” he adds, “feels like the Java language we already know, with a few augmentations.”

Si Dunn