BIG DATA: A well-written look at principles & best practices of scalable real-time data systems – #bookreview

 

 

Big Data

Principles and best practices of scalable real-time data systems

Nathan Marz, with James Warren

Manning – paperback

Get this book, whether you are new to working with Big Data or now an old hand at dealing with Big Data’s seemingly never-ending (and steadily expanding) complexities.

You may not agree with all that the authors offer or contend in this well-written “theory” text. But Nathan Marz’s Lambda Architecture is well worth serious consideration, especially if you are now trying to come up with more reliable and more efficient approaches to processing and mining Big Data. The writers’ explanations of some of the power, problems, and possibilities of Big Data are among the clearest and best I have read.

“More than 30,000 gigabytes of data are generated every second, and the rate of data creation is only accelerating,” Marz and Warren point out.

Thus, previous “solutions” for working with Big Data are now getting overwhelmed, not only by the sheer volume of information pouring in but by greater system complexities and failures of overworked hardware that now plague many outmoded systems.

The authors have structured their book to show “how to approach building a solution to any Big Data problem. The principles you’ll learn hold true regardless of the tooling in the current landscape, and you can use these principles to rigorously choose what tools are appropriate for your application.” In other words, they write, you will “learn how to fish, not just how to use a particular fishing rod.”

Marz’s Lambda Architecture also is at the heart of Big Data, the book. It is, the two authors explain, “an architecture that takes advantage of clustered hardware along with new tools designed specifically to capture and analyze web-scale data. It describes a scalable, easy-to-understand approach to Big Data systems that can be built and run by a small team.”

The Lambda Architecture has three layers: the batch layer, the serving layer, and the speed layer.

Not surprisingly, the book likewise is divided into three parts, each focusing on one of the layers:

  • In Part 1, chapters 4 through 9 deal with various aspects of the batch layer, such as building a batch layer from end to end and implementing an example batch layer.
  • Part 2 has two chapters that zero in on the serving layer. “The serving layer consists of databases that index and serve the results of the batch layer,” the writers explain. “Part 2 is short because databases that don’t require random writes are extraordinarily simple.”
  • In Part 3, chapters 12 through 17 explore and explain the Lambda Architecture’s speed layer, which “compensates for the high latency of the batch layer to enable up-to-date results for queries.”

Marz and Warren contend that “[t]he benefits of data systems built using the Lambda Architecture go beyond just scaling. Because your system will be able to handle much larger amounts of data, you’ll be able to collect even more data and get more value out of it. Increasing the amount and types of data you store will lead to more opportunities to mine your data, produce analytics, and build new applications.”

This book requires no previous experience with large-scale data analysis, nor with NoSQL tools. However, it helps to be somewhat familiar with traditional databases. Nathan Marz is the creator of Apache Storm and originator of the Lambda Architecture. James Warren is an analytics architect with a background in machine learning and scientific computing.

If you think the Big Data world already is too much with us, just stick around a while. Soon, it may involve almost every aspect of our lives.

Si Dunn

Programming for Musicians and Digital Artists: Creating Music with ChucK – #music #programming #bookreview

 

 

Programming for Musicians and Digital Artists

Creating Music with ChucK

Ajay Kapur, Perry Cook, Spencer Salazar and Ge Wang

Manning – paperback

Manning’s Programming for Musicians and Digital Artists is enjoyable, informative reading, particularly if you like music and programming and are motivated to combine them in some way.

The book offers plenty of clear how-to content for those who want to take their first deep dives into the techniques needed to make, modify and perform music using computers.

Indeed, this excellent guide can help take you from generating  “Hello, World” and “Twinkle, Twinkle, Little Star” to linking up with MIDI devices and creating sophisticated music and sounds that can be used in live performances and elsewhere.

Don’t be scared by the word “Programming” in the title. Yes, it can help–but it is not required–to have a little bit of programming experience. As you start working with the audio-centric programming language ChucK, you will simply type a few brief lines of code or paste them from downloaded files into a simple on-screen tool known as the “miniAudicle.” With this tool, you can then make changes and hear the results “instantly without interrupting other sounds being synthesized and heard,” the authors point out. You also can save your files, load different files and do other tasks quickly.

The free, open-source ChucK programming language, the authors’ emphasize, “is designed specifically for real-time sound synthesis and music creation.” Their book provides numerous short code examples to tinker with, as well as a few basic physics, math and music pointers that illustrate features and help support the authors’ descriptions.

Note: If your goal is to sit down at a keyboard and immediately start creating digital music, you may want to skip this book and look for other options. The authors concede that “many artists are happy with over-the-counter software systems and controllers for real-time performance work. And there are many who only want to use computers to produce static final products in the form of .wav/.mp3 files, CDs or collections of songs, sound tracks for videos, and more. A large number of those artists are happy to learn and use the packages and tools from commercial or free sources.

“But there are many, and we’re betting you’re one, who want more,” they add. “Maybe you’re coming to this book with a big idea (or many big ideas) and want the tools to help you realize it/them. Maybe you’re looking to shift directions in your art making. Or perhaps you already know how to program in a language such as Java, but you find it doesn’t do what you want.”

ChucK gives you “greater under-the-hood access” than some of the other popular music/sound languages and systems, such as Csound, SuperCollider, JSyn, Max/MSP and PD (Pure Data). And Chuck, the authors note, “is generally more succinct, requiring much less code (lines of typed text) than these other languages in order to accomplish a particular task.”

You learn how to work with many different tools, ranging from oscillators, to filters, to delay generators, reverberators and other audio effects, and MIDI (even without a MIDI interface and cable). You also learn how to generate the sounds of several different musical instruments.

ChucK has a key emphasis on ease of controlling time: for example, how long a tone or sounds occurs, how often it occurs within a set time period, and how long are the silences between tones or sounds.

I have not yet tried all of the code examples in the book, but the ones I have tried in several chapters have worked very well on a Windows laptop and are easily modified and tested in real time using the miniAudicle. (The book also shows how to install ChucK on Mac OS X and Ubuntu Linux systems).

Thus far, I have encountered only one typo in the printed book’s code examples. In Listing 1.8, “Playing notes with integer values,” there is a mistake in the line that is supposed to multiply the frequency of a tone pitch by 2. However, the line is printed “1 *=> myPitch;” — which simply repeats previous pitch. Changing the line to “2 *=> myPitch;” fixes the problem and takes only a couple of seconds to implement in the miniAudicle.

Si Dunn

 

JavaScript Application Design: A book you likely need if you are working with, or still learning, JavaScript – #programming #bookreview

JavaScript Application Design

A Build First Approach

Nicolas Bevacqua

Manning – paperback

 

I didn’t know how much I needed this book until I started reading it and exploring its code examples.

Many of us who have worked with JavaScript started our connections to the language in very haphazard fashions. We learned some of it on the job, under deadline pressure to fix or update somebody else’s code. Or we took an introductory class or two and then started picking up whatever else we could on the fly, including the bad habits of others around us who seemed to know a bit more about JavaScript than we knew at the moment.

Unfortunately, JavaScript is a big, messy programming language, and it offers numerous opportunities to crash and burn if you really don’t know what you are doing.

In his new book, JavaScript Application Design, Nicolas Bevacqua makes a compelling case for using “the Build First philosophy of designing for clean, well-structured, and testable applications before you write a single line of code.”

He writes: “You’ll learn about process automation, which will mitigate the odds of human error…. Build First is the foundation that will empower you to design clean, well-structured, and testable applications, which are easy to maintain and refactor. Those are the two fundamental aspects of Build First: process automation and design.”

In his well-written text, he argues: “Front-end development requires as much dedication to architecture planning and design as back-end development does. Long gone are the days when we’d copy a few snippets of code off the internet, paste them in our page, and call it a day. Mashing together JavaScript code as an afterthought no longer holds up to modern standards. JavaScript is now front and center.”

He continues: “We have many frameworks and libraries to choose from, which can help you organize your code by allowing you to write small components rather than a monolithic application. Maintainability isn’t something you can tack onto a code base whenever you’d like; it’s something you have to build into the application, and the philosophy under which the application is designed, from the beginning. Writing an application that isn’t designed to be maintainable translates into stacking feature after feature in an ever-so-slightly tilting Jenga tower.”

Bevacqua divides his nine-chapter book into just two parts: build processes and managing complexity. Here is how the chapters are organized:

  • PART 1: BUILD PROCESSES
    1 – Introduction to Build First
    2 – Composing build tasks and flows
    3 – Mastering environments and the development workflow
    4 – Release, deployment, and monitoring
  • PART 2: MANAGING COMPLEXITY
  • 5 – Embracing modularity and dependency management
    6 – Understanding asynchronous flow control methods in JavaScript
    7 – Leveraging the Model-View-Controller
    8 – Testing JavaScript components
    9 – REST API design and layered service architectures

Bevaqua notes that “Linting is often referred to as the first test you should set up when writing JavaScript. Where linters fail, unit tests come in.” He strongly pushes testing and automation right from the start.

Linting soon leads to Grunt, which Bevaqua uses as a task runner and key build tool (with selected modules) in this book. “Grunt is a tool that allows you to write, configure, and automate tasks–such as minifying a JavaScript file or compiling a LESS style sheet–for your application,” he states. (It also works well on Windows machines, which I find handy.)

Grunt leads to running a bit of Node.js on the command line. And if you’ve never worked with Node.js, Bevacqua takes the reader smoothly through the process of installing it and using it in linting exercises. Indeed, he devotes an entire appendix (B) to installing and running Grunt and picking the right plugins for the right tasks and targets.

One of the best parts of this book, to me, is how the author uses short code examples to introduce a concept, and  then builds upon the examples with helpful descriptions and more short but expanded code samples.

Nicolas Bevacqua offers his readers plenty of helpful how-to and why information. Using his book, I have begun applying the Build First approach to some new projects and learning to how test and automate more of my work. I feel as if I now have a good shot at getting a lot better at JavaScript.

There is one small but important glitch to note: At two points in my preview copy of the book from Manning, Bevacqua shows what he calls a simple way to create bare-minimum JSON manifest files. For example, echo “{}” > package.json. Creating a blank, starting-point manifest file did not work this way for me. Instead, I had to use echo {“name: ” “project-name”} > package.json. The empty package.json issue apparently is somehow related to certain versions of Node’s npm.

Si Dunn 

jQUERY UI IN ACTION: A smooth guide to getting, learning and using plugins supported by the jQuery Foundation – #bookreview

jQuery UI in Action

TJ VanToll

 (Manning – paperback)

 

TJ VanToll had two straightforward goals in mind when he decided to write this nicely prepared book: “I wanted to write about how to use the jQuery UI components in real-world usage scenarios and applications. I also wanted to tackle the tough questions for jQuery UI users. [Such as] Why should you use the jQuery UI datepicker instead of the native date picker included in HTML5? How do you use jQuery UI on mobile devices, especially in low bandwidth situations?”

According to the jQuery Foundation, “jQuery is a fast, small, and feature-rich JavaScript library. It makes things like HTML document traversal and manipulation, event handling, animation, and Ajax much simpler with an easy-to-use API that works across a multitude of browsers. With a combination of versatility and extensibility, jQuery has changed the way that millions of people write JavaScript.”

The problem with popularity, of course, is that jQuery became widely employed soon after it was introduced in 2006. Users quickly created a flood of jQuery plugins that, Van Toll writes, “had inconsistent APIs, and often had little or no documentation. Because of these problems, the jQuery team wanted to provide an official set of plugins in a centralized location. In September 2007 they created a new library with these plugins—jQuery UI.”

He adds: “From a high level, jQuery UI was, and still is, a collection of plugins and utilities that build on jQuery. But dig deeper and you find a set of consistent, well-documented, themeable building blocks to help you create everything from small websites to highly complex web applications. Unlike jQuery plugins, the plugins and utilities in jQuery UI are supported by the jQuery Foundation. You can count on them to be officially supported and maintained throughout the life of your application.”

Well-written and well-illustrated, jQuery UI in Action reflects VanToll’s knowledge and experience as a professional web developer and member of the core jQuery UI team.

The book is structured into three parts, encompassing 12 chapters. And it assumes readers have at least basic experience with JavaScript, CSS, and jQuery.

Part One’s chapters introduce jQuery UI and “the ins and outs of widgets…the core building blocks of jQuery UI.”

Part Two’s chapters offer “a comprehensive look at the components of jQuery UI: twelve jQuery UI widgets (chapters 3–4), five jQuery
UI interactions (chapter 5), numerous jQuery UI effects (chapter 6), and the jQuery UI CSS framework (chapter 7).” VanToll explains how each component works and shows how to apply the knowledge to real-world applications. The example projects include: building complex webforms with jQueryUI; using layout and utility widgets; adding interaction to interfaces; and using built-in and customized themes to provide “a consistent look to all widgets.”

Part Three focuses on “Customization and advanced usage.” Here, VanToll explores such topics as using the widget factory to create custom widgets, preparing applications for production, and building a flight-search application “at real-world scale.” In the final chapter, he takes us under jQuery’s hood “to dig into a series of utilities, methods, and properties intended for more advanced usage of the library.”

If you work with jQuery or are ready to start using it, take a good look at jQuery UI, as well. As this book promises, “You’re only one tag away from richer user interfaces….” That tag is pretty simple: <script src=”jquery-ui.js”> — but a lot can happen once you include it.

TJ VanToll’s new book should be your go-to guide for getting, learning and putting jQuery UI into action.

Si Dunn

 

 

 

 

 

http://amzn.to/1r1VwUI

 

You’ll master jQuery UI’s five main interactions—draggable, droppable, resizable, selectable, and sortable—and learn UI techniques that work across all devices.

HADOOP IN PRACTICE, 2nd Edition – An updated guide to handling some of the ‘trickier and dirtier aspects of Hadoop’ – #programming #bookreview

 

Hadoop in Practice, Second Edition

Alex Holmes

(Manning – paperback )

 

The Hadoop world has undergone some big changes lately, and this hefty, updated edition offers excellent coverage of a lot of what’s new. If you currently work with Hadoop and MapReduce or are planning to take them up soon, give serious consideration to adding this well-written book to your technical library. A key feature of the book is its “104 techniques.” These show how to accomplish practical and important tasks when working with Hadoop, MapReduce and their growing array of software “friends.”

The author, Alex Holmes, has been working with Hadoop for more than six years and is a software engineer, author, speaker, and blogger specializing in large-scale Hadoop projects.

His new second edition, he writes, “covers Hadoop 2, which at the time of writing is the current production-ready version of Hadoop. The first edition of the book covered Hadoop 0.22 (Hadoop 1 wasn’t yet out), and Hadoop 2 has turned the world upside-down and opened up the Hadoop platform to processing paradigms beyond MapReduce. YARN, the new scheduler and application manager in Hadoop 2, is complex and new to the community, which prompted me to dedicate a new chapter 2 to covering YARN basics and to discussing how MapReduce now functions as a YARN application.”

In the book, Holmes notes that “Parquet has also recently emerged as a new way to store data in HDFS—its columnar format can yield both space and time efficiencies in your data pipelines, and it’s quickly becoming the ubiquitous way to store data. Chapter 4 includes extensive coverage of Parquet, which includes how Parquet supports sophisticated object models such as Avro and how various Hadoop tools can use Parquet.”

Furthermore, “[h]ow data is being ingested into Hadoop has also evolved since the first edition,” Holmes points out, “and Kafka has emerged as the new data pipeline, which serves as the transport tier between your data producers and data consumers, where a consumer would be a system
such as Camus that can pull data from Kafka into HDFS. Chapter 5, which covers moving data into and out of Hadoop, now includes coverage of Kafka and Camus.”  [Reviewer’s note: Interesting software names here. Franz Kafka and Alfred Camus were writers deeply concerned about finding clarity and meaning in a world that seemed to offer none.]

Holmes adds that “[t]here are many new technologies that YARN now can support side by side in the same cluster, and some of the more exciting and promising technologies are covered in the new part 4, titled ‘Beyond MapReduce,’ where I cover some compelling new SQL technologies such as Impala and Spark SQL. The last chapter, also new for this edition, looks at how you can write your own YARN application, and it’s packed with information about important features to support your YARN application.”

Hadoop and MapReduce have gained reputations (well-earned) for being difficult to set up, use and master. In his new edition, Holmes describes his own early experiences: “The greatest challenge we faced when working with Hadoop, and specifically MapReduce, was relearning how to solve problems with it. MapReduce is its own flavor of parallel programming, and it’s quite different from the in-JVM programming that we were accustomed to. The first big hurdle was training our brains to think MapReduce, a topic which the book Hadoop in Action by Chuck Lam (Manning Publications, 2010) covers well.”

(These days, of course, there are both open source and commercial releases of Hadoop, as well as quickstart virtual machine versions that are good for learning Hadoop.)

Holmes continues: “After one is used to thinking in MapReduce, the next challenge is typically related to the logistics of working with Hadoop, such as how to move data in and out of HDFS and effective and efficient ways to work with data in Hadoop. These areas of Hadoop haven’t received much coverage, and that’s what attracted me to the potential of this book—the chance to go beyond the fundamental word-count Hadoop uses and covering some of the trickier and dirtier aspects of Hadoop.”

If you have difficulty explaining Hadoop to others (such as a manager or executive hesitant to let it be implemented), Holmes offers a succint summation in his updated book:

“Doug Cutting, the creator of Hadoop, likes to call Hadoop the kernel for big data, and I would tend to agree. With its distributed storage and compute capabilities, Hadoop is fundamentally an enabling technology for working with huge datasets. Hadoop provides a bridge between structured (RDBMS) and unstructured (log files, XML, text) data and allows these datasets to be easily joined together.”

One book cannot possibly cover everything you need to know about Hadoop, MapReduce, Parquet, Kafka, Camus, YARN and other technologies. And this book and its software examples assume that you have some experience with Java, XML and JSON. Yet Hadoop in Practice, Second Edition gives a very good and reasonably deep overview, spanning such major categories as background and fundamentals, data logistics, Big Data patterns, and moving beyond MapReduce.

Si Dunn

 

 

Advanced Software Testing, Vol. 2, 2nd Edition – Study guide for ISTQB Advanced Test Manager – #bookreview

Advanced Software Testing, Volume 2, 2nd Edition

Guide to the ISTQB Advanced Certification as an Advanced Test Manager

Rex Black

(Rocky Nook – paperback)

 

Software testing is a complex and constantly evolving field. And having some well-recognized certifications is a good way to help encourage  your continued employability as a software tester and manager of software test teams.

Advanced Software Testing, Volume 2, 2nd Edition, focuses on showing you how to obtain an International Software Testing Qualifications Board (ISTQB) certification as an advanced test manager. The 519-page book is well-written and lays out what test managers should know to gain advanced skills in test estimation, test planning, test monitoring, and test control.

It also emphasizes  knowing how to define overall testing goals and strategies for the systems you and your team are testing. And it gives you strategies for preparing for and passing the 65-question Advanced Test Manager qualification test that is administered by ISTQB member boards and exam providers.

This second edition has been updated to reflect the ISTQB’s Advanced Test Manager 2012 Syllabus.  Advanced Software Testing, Volume 2, 2nd Edition takes a hands-on, exercise-rich approach, and it provides experience with essential how-tos for planning, scheduling, and tracking important tasks.

The updated book focuses on a variety of key processes that a software test manager must be able to handle, including describing and organizing the activities necessary to select, find and assign the right number of resources for testing tasks. You also must learn how to organize and lead testing teams, and how to manage the communications among testing teams’ members and between testing teams and all the other stakeholders. And you will need to know how to justify your testing decisions and report necessary information both to your superiors and members of your teams.

As for taking the complicated qualifications test, the author urges: “Don’t panic! Remember, the exam is meant to test your achievement of the learning objectives in the Advanced  Test Manager syllabus.” In other words, you cannot simply skim this book and take the exam. You must spend significant time on the learning exercises, sample questions and ISTQB glossary.

Si Dunn

***

Get the book here: Advanced Software Testing, Volume 2, 2nd Edition

***

Photoshop CC and Lightroom – An elegant, well-focused how-to handbook from Rocky Nook – #photography #bookreview

 

Photoshop CC and Lightroom

A Photographer’s Handbook

Stephen Laskevitch

(Rocky Nook – paperback, Kindle)

 

Stephen Laskevitch’s Photoshop CC and Lightroom is an excellent how-to book that both instructs and inspires.

This elegant new how-to book from Rocky Nook is aimed at digital photographers and graphics designers “who want to learn the basic tools and image editing steps within Photoshop and Lightroom to recreate professional looking images.” However, the book also is recommended for “a wide range of technicians and office workers who simply want to do more effective image editing.”

As a sometimes-photographer and not-frequent-enough user of feature-rich Photoshop, I definitely need how-to books like this to keep me on track with the features that I “know,” while also reminding me that there are many useful features I have not yet tried or learned. Fortunately, Laskevitch, an Adobe Certified Instructor, deliberately avoids the common tendency to showcase just the  “wow-factor Photoshop techniques.” Instead, he emphasizes “all the key techniques for good image editing: using layers and layer blending, color correction, printer profiles, and more.”

His book is richly illustrated with photographs that can inspire you to pick up your camera and go shoot. And it has plenty of how-to illustrations and steps for using the 2014 release of Photoshop CC, plus its companions: Bridge, Camera Raw, and Lightroom 5, as you process, enhance and preserve your images.

Bridge is a tool that lets you examine, sort, rate and organize the images in a folder. Adobe Camera Raw provides a few settings that can be selected or adjusted, and Laskevitch recommends shooting in RAW format, unless shooting snapshots. “One of the biggst advantages of RAW files,” he emphasizes, “is that they have more than 8 bits per channel of information and can therefore be edited more than JPEG files.” Lightroom, meanwhile, is “a photographer-friendly database application” that helps you keep track of your images and where you have stored them.

Photoshop CC and Lightroom is divided into two parts and ten chapters:

The Setup

  • Important Terms & Concepts
  • System Configuration
  • The Interface: A Hands-On Tour

The Workflow

  • Capture & Import
  • Organizing & Archiving Images
  • Global Adjustments
  • Local Adjustments
  • Cleaning & Retouching
  • Creative Edits & Alternatives
  • Output

“Output,” Laskevitch notes, “is the creation of what I call deliverables, whether that is a print, a book, a web site, or a digital file. Printing should be easier than it is, especially after all of these years of digital imaging. Improved with each release of Photoshop, the method I outline is simpler than ever. But since it uses profiles tht describe your printer’s characteristics to achineve stunning consistency and optimal results, you’ll have to keep focused nonetheless. This method,” he explains,” can also allaow you to experiment with many more papers than your printer manufacturer supplies.”

You do not have to have any prior Photoshop experience to benefit from Photoshop CC and Lightroom: A Photographer’s Handbook. And Photoshop works with either Windows or Mac computers, the author points out. Also, many (but not all) of the worfklows and techniques he describes can be used with previous versions of the software products, as well.

Si Dunn