Posts

Showing posts with the label scala

Download All Project dependencies using Maven

At-times there is need to physically download all the dependencies for Java/Scala based applications. Especially when we want to develop or run application in a remote offline environments etc. Maven has this convenience plug-in to rescue us:

Binary Data to Float using Spark SQL

Often when there is massive data communication among the enterprise applications, the data is converted to binary datatype, for reasons like: Faster I/O, Smaller Size, any type of data can be represented, etc. This brings in need to convert binary back to known/readable data type for further analysis/processing etc. For example, here floating point data is represented in Decimal Array[Byte] then stored in Binary format. Ex: [62, -10, 0, 0] Whole floating point equivalent is: 0.480 To achieve above conversion using Apache Spark Sql DataFrames: Voila!

JDBC Operations in Spark + Scala

Here is the sample code to create a sample Dataframe from a RDD and then insert that into MySql database. This approved should work for any other relational databases. Happy Sparking...