Release Notes - Giraph - Version 1.0.0

Sub-task

[GIRAPH-206] - Break out SimpleShortestPathVertex
[GIRAPH-360] - Keep track of the task id in ChannelRotater to send requests without knowing the worker id upfront

Bug

[GIRAPH-52] - There should be a scheme to limit the counter
[GIRAPH-130] - Fix Javadoc warnings
[GIRAPH-148] - giraph-site.xml needs Apache header
[GIRAPH-151] - IntIntNullIntVertex.initialize method should handle "null" edges argument
[GIRAPH-161] - Handling null messages and edges when initializing IntIntNullIntVertex
[GIRAPH-162] - BspCase.setup() should catch FileNotFoundException thrown from org.apache.hadoop.fs.FileSystem.listStatus()
[GIRAPH-164] - fix 5 "Line is longer than 80 characters" style errors in GiraphRunner
[GIRAPH-184] - Upgrade to junit4
[GIRAPH-198] - running Giraph trunk on Hadoop 2.0.0-alpha leads to an exception
[GIRAPH-199] - HADOOP_HOME has been deprecated in Hadoop 1.X and Hadoop 2.X
[GIRAPH-205] - Move Giraph jar to root level of tar.gz
[GIRAPH-209] - Include munge version in artifact name
[GIRAPH-210] - Hadoop 1.0 profile has no activation
[GIRAPH-212] - Security is busted since GIRAPH-168
[GIRAPH-214] - GiraphJob should have configuration split out of it to be cleaner (GiraphConf)
[GIRAPH-216] - NullWritable as VertexData, EdgeData or MessageData should be allowed
[GIRAPH-217] - Add SimpleTriangleClosingVertex to Giraph examples
[GIRAPH-218] - Consolidate all I/O Format classes under one roof in lib/ directory
[GIRAPH-219] - pom in giraph-formats-contrib should have groupId 'org.apache.giraph'.
[GIRAPH-223] - Need to put Giraph jar on classpath, post-GIRAPH-205.
[GIRAPH-227] - ArrayWritables and their sub-classes don't play well with GiraphJob.run()
[GIRAPH-228] - SimpleTriangleClosingVertex should not use ArrayWritable for a vertex value
[GIRAPH-233] - Small errors found by FindBugs
[GIRAPH-241] - Small typos in var names in NettyWorkerClient.java
[GIRAPH-258] - Check type compatibility before submitting job
[GIRAPH-267] - Jobs can get killed for not reporting status during INPUT SUPERSTEP
[GIRAPH-274] - Jobs still failing due to tasks timeout during INPUT_SUPERSTEP
[GIRAPH-276] - Fix broken tests in pseudo-distributed mode
[GIRAPH-308] - Giraph consistently creates 10% more InputSplits than one would expect
[GIRAPH-318] - New Iterator in LocalityInfoSorter is not working
[GIRAPH-347] - GiraphConfiguration broke hcatalog build
[GIRAPH-350] - HBaseVertex i/o formats are not being injected with Configuration via Configurable interface
[GIRAPH-353] - Received metrics are not thread-safe
[GIRAPH-368] - HBaseVertexInput/Output formats handle setConf() internally
[GIRAPH-369] - bin/giraph broken
[GIRAPH-370] - AccumuloVertexOutputFormat public visibility for TABLE_NAME
[GIRAPH-378] - Giraph's tar.gz distributable build does not include the bin/ directory (or the run scripts!) any more
[GIRAPH-383] - giraph-formats-contrib test failures for -Phadoop_0.23
[GIRAPH-389] - Multithreading should intelligently allocate the thread pools
[GIRAPH-392] - Can't build hadoop 0.23+ profiles
[GIRAPH-442] - FileNotFoundException: <baseDirectory>/_zkServer is checked before it is created
[GIRAPH-477] - Fetching locality info in InputSplitPathOrganizer causes jobs to hang
[GIRAPH-524] - Giraph can receive input from vertex or edge-centric data sets; its output is graph data, not "vertices"
[GIRAPH-573] - Giraph is ready for port to Mesos or other cluster frameworks
[GIRAPH-576] - BspServiceMaster.failureCleanup() shouldn't pass null in observers' applicationFailed() method
[GIRAPH-599] - Hive IO dependency issues with some Hadoop profiles
[GIRAPH-626] - Remove use of deprecated API's in giraph-core
[GIRAPH-630] - Convergence detection broken in o.a.g.examples.PageRankVertex
[GIRAPH-634] - InternalGiraphRunner with TestGraph is broken

Improvement

[GIRAPH-20] - Move temporary test files from the project directory
[GIRAPH-85] - Simplify return expression in RPCCommunications::getRPCProxy
[GIRAPH-117] - DefaultWorkerContext should preserve the method signatures of WorkerContext
[GIRAPH-133] - Typo in JavaDoc in BspCase::remove
[GIRAPH-136] - Error message for bin/giraph could be improved
[GIRAPH-137] - De-duplicate pagerank implementation in PageRankBenchmark
[GIRAPH-139] - Change PageRankBenchmark to be accessible via bin/giraph
[GIRAPH-142] - _hadoopBsp should be prefixable via configuration
[GIRAPH-145] - Change partition request log level to debug rather than info
[GIRAPH-156] - Users should be able to set simple 'custom arguments' via org.apache.giraph.GiraphRunner
[GIRAPH-175] - Replace manual array copy to utility method call
[GIRAPH-176] - BasicRPCCommunications has unnecessary cast of Vertex
[GIRAPH-177] - SimplePageRankVertex has two redundant casts
[GIRAPH-179] - BspServiceMaster's PathFilter can be simplified
[GIRAPH-181] - Add Hadoop 1.0 profile to pom.xml
[GIRAPH-183] - Add Claudio's FOSDEM presentation (slides and video) to the site
[GIRAPH-185] - Improve concurrency of putMsg / putMsgList
[GIRAPH-186] - Improve concurrency of putVertexList
[GIRAPH-188] - Merge inMessages and vertex message list
[GIRAPH-192] - Move aggregators to a separate sub-package
[GIRAPH-200] - remove hadoop RPC and keep just netty
[GIRAPH-211] - Add secure authentication to Netty IPC
[GIRAPH-213] - NettyClient.stop() could deadlock according to netty.io docs
[GIRAPH-220] - Default implementation of BasicVertex#sendMsgToAllEdges()
[GIRAPH-225] - Guava version in POM.XML is really old. Updated to version 12.0
[GIRAPH-237] - Add a .gitignore file with some common defaults for java projects
[GIRAPH-246] - Periodic worker calls to context.progress() will prevent timeout on some Hadoop clusters during barrier waits
[GIRAPH-247] - Introduce edge based partitioning for InputSplits
[GIRAPH-248] - Generic IdentityVertex for IO testing
[GIRAPH-250] - Let workers contending for InputSplits during INPUT_SUPERSTEP guess better, choose quicker.
[GIRAPH-252] - Detecting Number of Vertices for inputs without an entry for vertices with no out edge
[GIRAPH-254] - Constant Variable name misspelled in HashMasterPartitioner.java
[GIRAPH-255] - Local WritableRequest variable name misspelled as 'writableReauest' several times.
[GIRAPH-256] - Partitioning outgoing graph data during INPUT_SUPERSTEP by # of vertices results in wide variance in RPC message sizes
[GIRAPH-261] - Rename isQuiet variable
[GIRAPH-275] - Restore data locality to workers reading InputSplits where possible without querying NameNode, ZooKeeper
[GIRAPH-286] - Remove DISCLAIMER from source tree
[GIRAPH-287] - Add option to limit the number of open requests
[GIRAPH-299] - Constant value in BspServiceMaster should be static
[GIRAPH-301] - InputSplit Reservations are clumping, leaving many workers asleep while other process too many splits and get overloaded.
[GIRAPH-305] - Adding an argument to GiraphRunner for Master Compute classes
[GIRAPH-307] - InputSplit list can be long with many workers (and locality info) and should not be re-created every time a worker calls reserveInputSplit()
[GIRAPH-314] - Implement better message grouping to improve performance in SimpleTriangleClosingVertex
[GIRAPH-322] - Run Length Encoding for Vertex#sendMessageToAllEdges might curb out of control message growth in large scale jobs
[GIRAPH-328] - Outgoing messages from current superstep should be grouped at the sender by owning worker, not by partition
[GIRAPH-336] - style violations in giraph-formats-contrib
[GIRAPH-338] - More Rat Ignores
[GIRAPH-351] - Fail job early when there is no input
[GIRAPH-367] - Expose WorkerInfo to clients
[GIRAPH-459] - Group Vertex Mutations by Partition ID
[GIRAPH-480] - Add convergence detection to org.apache.giraph.examples.RandomWalkVertex
[GIRAPH-568] - Giraph on YARN will need a WebUI to display job stats, Yammer metrics, whatever
[GIRAPH-570] - Create YARN RPC Records using BuilderUtils instead of populating them by hand
[GIRAPH-612] - Improve website for upcoming release
[GIRAPH-617] - Website Documentation: Vertex/Edge Input/Output
[GIRAPH-631] - Remove Hardcoded Dependency on Hadoop-2.0.3-alpha from YARN and replace with a more flexible Maven config
[GIRAPH-632] - YARN dependencies add a fair amount of size to the fat jar and may be subject to simplification
[GIRAPH-633] - Rename VertexEdges to OutEdges

New Feature

[GIRAPH-4] - New project logo
[GIRAPH-81] - Create annotations on provided algorithms for cli
[GIRAPH-127] - Extending the API with a master.compute() function.
[GIRAPH-143] - Add support for giraph to have a conf file
[GIRAPH-160] - Vertex reader that reads adjacency lists with no vertex and edge values associated
[GIRAPH-295] - Additional Example Algorithm to compute Outdegree and Indegree
[GIRAPH-312] - Giraph needs an admin script

Task

[GIRAPH-236] - Add FindBugs to maven build

Test

[GIRAPH-26] - Improve PseudoRandomVertexInputFormat to create a more realistic synthetic graph (e.g. power-law distributed vertex-cardinality).

Wish

[GIRAPH-317] - Add subpackages to comm