Welcome!

Search Authors: David Weinberger, Adrian Bridgwater, Mary Ellen Power, Gathering Clouds, Alex Forbes

Related Topics: Cloud Expo, Open Source

Cloud Expo: Blog Feed Post

Faster R in Hadoop: rmr 1.3 Now Available

Continuing the Big Data integration of R and Hadoop
By David Smith
Article Rating:
July 26, 2012 04:00 AM EDT
Reads:
 2,554

The RHadoop project continues the Big Data integration of R and Hadoop, with a new update to its rmr package.

Version 1.3 of rmr improves the performance of map-reduce jobs for Hadoop written in R. New features include: An optional vectorized API for efficient R programming when dealing with small records.

Fast C implementations for serialization and deserialization from and to typedbytes. Other readers and writers work much better in vectorized mode, namely csv and text Additional steps to support structured data better (use more data frames and fewer lists in the API) More forgiving behavior for package loading and bug fixes Also, the documentation has gotten a major overhaul in this version, with pages of combined text, code and graphics generated automatically using the knitr package.




(RHadoop lead developer Antonio Piccolboni provides some background on how knitr is used in these documentation guidelines.)

If you haven't take a look at rmr before, this tutorial by Jeffrey Breen is a great place to get started. Otherwise, check out the wiki pages on the RHadoop github site, linked below. github: RevolutionAnalytics / RHadoop

Published July 26, 2012 – Reads 2,554
Copyright © 2012 SYS-CON Media, Inc. — All Rights Reserved.
Syndicated stories and blog feeds, all rights reserved by the author.

More Stories By David Smith

David Smith is Vice President of Marketing and Community at Revolution Analytics. He has a long history with the R and statistics communities. After graduating with a degree in Statistics from the University of Adelaide, South Australia, he spent four years researching statistical methodology at Lancaster University in the United Kingdom, where he also developed a number of packages for the S-PLUS statistical modeling environment. He continued his association with S-PLUS at Insightful (now TIBCO Spotfire) overseeing the product management of S-PLUS and other statistical and data mining products.<

David smith is the co-author (with Bill Venables) of the popular tutorial manual, An Introduction to R, and one of the originating developers of the ESS: Emacs Speaks Statistics project. Today, he leads marketing for REvolution R, supports R communities worldwide, and is responsible for the Revolutions blog. Prior to joining Revolution Analytics, he served as vice president of product management at Zynchros, Inc. Follow him on twitter at @RevoDavid

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.