NoSQL database news, reviews and tutorials - Go NoSql

Wordnik’s Experience with MongoDB

Erik Howard — Mon, 25 Oct 2010 23:03:26 +0000

Wordnik is an unconventional dictionary who’s goal is to show you as much information as possible, as fast as they can find it, for every word in English.

Wordnik is a heavy user of 10Gen’s nosql database MongoDB. They just recently blogged about their 1 year anniversary of moving from MySql to MongoDB. Some of the statistics include:

MongoDB servers an average of 500k requests/per hour. With 4x that number during peak periods
More than 12 billion documents stored by MongoDB
Around 3 TB storage per node
Easily sustained insert speeds of 8k documents/second, often with bursts to 50k documents/second
Every retrieval type is faster than MySql
- example fetch time reduced from 400ms to 60ms
- dictionary entries from 20ms to 1ms
- document metadata from 30ms to .1ms
- spelling suggestions from 10ms to 1.2ms

Mongomatic, A New MongoDB Ruby Library

Erik Howard — Fri, 13 Aug 2010 18:43:20 +0000

Mongomatic, from Ben Myles, is a new Ruby library for accessing MongoDB documents.

The primary design goals of Mongomatic are simplicity and adherence to MongoDB conventions. Mongomatic is not an ActiveRecord clone and has been designed to fit nicely with the document-oriented nature of MongoDB.

The library requires ActiveSupport (2 or 3) and the Ruby MongoDB driver. Ruby versions 1.8.x and 1.9.x are supported. Other versions of Ruby may be supported but have not been tested.

Mongomatic can be installed from GitHub or via RubyGems.

GitHub

git clone http://github.com/benmyles/mongomatic.git

Ruby Gems

gem install mongomatic

CouchDB now available on the Google Android platform

Erik Howard — Wed, 11 Aug 2010 18:03:28 +0000

Couchio , corporate sponsor of the CouchDB NoSQL database CouchDB, announced the first developer preview of the CouchDB SDK for Android devices.

According to Couchio, CouchDB on Android lets you build web-apps or native apps that take full advantage of CouchDB’s built-in, reliable peer-to-peer sync facilities.

Our goal is to provide you with a kick-ass SDK to build web- and native applications using CouchDB as the device-native data store. You will be able to use all your hard earned knowledge about web technologies to build apps on mobile devices.

This release does come with a few caveats. Namely the build is not yet optimized for performance on mobile devices, memory consumption or installation footprint. Couchio does say that they are working on this and many other areas in a future release.

The source code is available on GitHub. If you are on an Android device, you can install CouchDB SDK for Android directly from the Android Marketplace.

Redis Cookbook

Erik Howard — Fri, 09 Apr 2010 02:32:31 +0000

Check-0ut the new community-powered Redis Cookbook for recipes, how-to’s and more. The source is available on GitHub for you to fork and add your own contributions.

CouchDB vs MongoDB Comparison

Erik Howard — Tue, 30 Mar 2010 18:01:31 +0000

Great presentation on the usage of the two most well known NoSql document databases. CouchDB and MongoDB. Also an excellent explanation of map/reduce.

CouchDB Vs MongoDB

View more presentations from Gabriele Lana.

How to install Voldemort on Windows

Erik Howard — Fri, 26 Mar 2010 04:17:19 +0000

Project Voldemort is a scalable, highly available, NoSql, distributed, key/value store developed by Linkedin which in turn was inspired by Amazon’s Dynamo paper. The source code for Voldemort is currently hosted at GitHub.

Voldemort is written in Java and will run on any platform where Java can run. Getting Voldemort up and running on Linux/OSX or any other Posix based system is seamless. Running Voldemort is a little challenging, but not impossible.

Voldemort Prerequisites

Recent Java JDK with Server HotSpot VM
Apache Ant
Optionally Python 2.6

Download and install the Java SDK and Apache Ant. Make sure the Apache Ant bin directory is on your path.

To verify that you have the Server HotSpot VM installed, open a command window and type the following:

java -server -version

You should see something like this:

java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04)
Java HotSpot(TM) Server VM (build 10.0-b22, mixed mode)
java version "1.6.0_17"Java(TM) SE Runtime Environment (build 1.6.0_17-b04)Java HotSpot(TM) Server VM (build 10.0-b22, mixed mode)

If you get an error about no ‘server’ JVM at c:\Program Files\Java\jreXXX\bin\server.jvm.dll, then you’ll need to monkey-patch your Java installation before continuing.

Java Monkey Patch

Locate the directory where your Java JDK was installed (eg C:\Program Files\Java\jdk1.6.0_06)
Copy the \jre\bin\server folder to JRE\bin folder (eg: C:\Program Files\Java\jre6\bin)

Verify your Java installation again by entering the following in a command window:

java -server -version

You should get output similar to the paragraph above.

Installing Voldemort

For this tutorial, download the latest stable release from the Voldemort repository at GitHub. For this tutorial, I used Release 0.80.1.

After the download is complete, extract the files to a directory of your choosing.

If you navigate to the bin directory, you’ll notice that the majority of script files are shell scripts. Well that’s not going to work on Windows. Time for some more monkey patching.

Shell Script Monkey Patching

For this tutorial I will convert the three scripts you need to get Voldemort up and running:

run-class.sh
voldemort-server.sh
voldermort-shell.sh

Copy or download the 3 files below and place them in your Voldemort \bin directory.

run-class.bat [Download from GitHub]

@echo off

REM

REM Licensed under the Apache License, Version 2.0 (the “License”);

REM you may not use this file except in compliance with the License.

REM You may obtain a copy of the License at

REM

REM http://www.apache.org/licenses/LICENSE-2.0

REM

REM Unless required by applicable law or agreed to in writing, software

REM distributed under the License is distributed on an “AS IS” BASIS,

REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

REM See the License for the specific language governing permissions and

REM limitations under the License.

REM

set Count=0

for %%a in (%*) do set /a Count+=1

if %Count% gtr 1 goto :continue

echo %0 java-class-name [options]

goto :eof

:continue

SET BASE_DIR=%~dp0..

SET CLASSPATH=.

for %%j in (%BASE_DIR%\dist\*.jar) do (call :append_classpath “%%j”)

for %%j in (%BASE_DIR%\lib\*.jar) do (call :append_classpath “%%j”)

goto :run

:append_classpath

set CLASSPATH=%CLASSPATH%;%1

goto :eof

:run

REM You might have to adjust the max allowable memory setting to fit your environment

if “%VOLD_OPTS%” == “” set “VOLD_OPTS=-Xmx256M -server -Dcom.sun.management.jmxremote”

java -Dlog4j.configuration=src/java/log4j.properties %VOLD_OPTS% -cp %CLASSPATH% %*

endlocal

:eof

@echo off
REMREM Copyright 2008-2009 LinkedIn, IncREM Copyright 2010 Erik HowardREMREM Licensed under the Apache License, Version 2.0 (the “License”);REM you may not use this file except in compliance with the License.REM You may obtain a copy of the License atREMREM http://www.apache.org/licenses/LICENSE-2.0REMREM Unless required by applicable law or agreed to in writing, softwareREM distributed under the License is distributed on an “AS IS” BASIS,REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.REM See the License for the specific language governing permissions andREM limitations under the License.REM
set Count=0for %%a in (%*) do set /a Count+=1if %Count% gtr 1 goto :continueecho %0 java-class-name [options]goto :eof:continue
SET BASE_DIR=%~dp0..SET CLASSPATH=.
for %%j in (%BASE_DIR%\dist\*.jar) do (call :append_classpath “%%j”)for %%j in (%BASE_DIR%\lib\*.jar) do (call :append_classpath “%%j”)goto :run
:append_classpathset CLASSPATH=%CLASSPATH%;%1goto :eof
:runREM You might have to adjust the max allowable memory setting to fit your environmentif “%VOLD_OPTS%” == “” set “VOLD_OPTS=-Xmx256M -server -Dcom.sun.management.jmxremote”java -Dlog4j.configuration=src/java/log4j.properties %VOLD_OPTS% -cp %CLASSPATH% %*
endlocal:eof

voldemort-shell.bat [Download from GitHub]

@echo off REM REM Copyright 2008-2009 LinkedIn, Inc REM Copyright 2010 Erik Howard REM REM Licensed under the Apache License, Version 2.0 (the "License"); REM you may not use this file except in compliance with the License. REM You may obtain a copy of the License at REM REM http://www.apache.org/licenses/LICENSE-2.0 REM REM Unless required by applicable law or agreed to in writing, software REM distributed under the License is distributed on an "AS IS" BASIS, REM WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. REM See the License for the specific language governing permissions and REM limitations under the License. REM set Count=0 for %%a in (%*) do set /a Count+=1 if %Count% equ 2 goto :continue echo USAGE: bin/voldemort-shell.bat store_name bootstrap_url [command_file] goto :eof :continue setlocal SET BASE_DIR=%~dp0.. %BASE_DIR%\bin\run-class.bat jline.ConsoleRunner voldemort.VoldemortClientShell %* endlocal :eof

voldemort-server.bat [Download from GitHub]

Starting Voldemort

From a command prompt, go to the directory where you extracted Voldemort and type the following:

bin\voldemort-server.bat config\single_node_cluster

You should see several lines of information scroll by. Congratulations – you’ve just started Project Voldemort on Windows!

Testing Voldemort

Open up another command window and change to your Voldemort project directory.

Type the following to startup the Voldemort shell:

bin\voldemort-shell.bat test tcp://localhost:6666

With the shell up and running, lets try out a few commands


put "name" "erik"

get "name"

version(0:1) "erik"

delete "name"

get "name"

null

exit

k k thx bye

Conclusion

Before you can use the full functionality of Voldemort, you must first convert the remaining shell scripts in the bin directory to dos batch scripts. Good Luck.

MongoDB 1.4 is production ready

Erik Howard — Thu, 25 Mar 2010 23:38:00 +0000

MongoDB

Today 10Gen, the creators of MongoDB, announced in a blog post that MongoDB 1.4 is production ready.

Some of the new features and enhancements include:

Core server enhancements

indexing memory improvements
background index creation
better detection of regular expressions so the index can be used in more cases

Replication and Sharding

better handling for restarting slaves offline for a while
fast new slaves from snapshot
$inc replication fixes
2 phase commit on config servers

Deployment and production

ability to do fsync + lock for backing up raw files
option for separate directory per database
REST interface is off by default
rotate logs with the db command logRotate
new mongostat tool and db.serverStatus enhancements

Query language improvements

$all with regex
$not
partial matching for array elements $elemMatch
$ operator for updating arrays
$addToSet
$unset
$pull supports matching objects
$set with array indices

Geo

2d geospatial search
geo $center and $box searches

For more information, read the full release notes here.

Hadoopifying your backend

Erik Howard — Wed, 24 Mar 2010 18:25:00 +0000

If you have a legacy system and your thinking about migrating to Hadoop, you need to read BackType Technology’s post entitled “Migrating data from a SQL database to Hadoop“.

Experiment with MongoDB in your browser

Erik Howard — Wed, 24 Mar 2010 06:43:27 +0000

Try MongoDB Now

Wanted to try out MongoDB, but have no time to install and configure it? Well head over to http://try.mongodb.org/ and play around with MongoDB within your browser.

10gen Announces MongoSF Conference

Erik Howard — Fri, 19 Mar 2010 05:13:05 +0000

10Gen MongoDB

Full-day April 30th event in San Francisco to explore development on the NoSql document database MongoDB

The conference features sessions on database internals, schema design, GridFS, replication, sharding, and more. In addition to these topics, attendees can learn about MongoDB in the real world through a series of presentations about production deployments at Justin.tv, Boxed Ice, Punchbowl Software, and Harmony App.

Small, hands-on, language-specific development workshops will take place alongside the presentation tracks. The Conference Center Lounge will also be available for the duration of the conference for hacking, networking, and discussion.

DATE: Friday April 30, 2010

TIME: 8:30am – 5:30pm

LOCATION: Bently Reserve, 400 Sansome Street, San Francisco, CA 94111

BUY TICKETS: Register