Skip navigation.

Pas Apicella

Syndicate content
Information on Pivotal -> Product Family - GemFire, SQLFire/GemFireXD, tc Server, RabbitMQ, Pivotal HD, Greenplum DB and SpringPas Apicellahttp://www.blogger.com/profile/09389663166398991762noreply@blogger.comBlogger270125
Updated: 1 hour 36 min ago

Dept/Emp POJO's with sample data for Pivotal GemFire

Tue, 2014-08-12 21:57
I constantly blog about using DEPARTMENT/EMPLOYEE POJO'S with sample data. Here is how to create a file with data to load into GemFire to give you that sample set.

Note: You would need to create POJO'S for Department/Empployee objects that have getter/setter for the attributes mentioned below.

Dept Data

put --key=10 --value=('deptno':10,'name':'ACCOUNTING') --value-class=pivotal.au.se.deptemp.beans.Department --region=departments;
put --key=20 --value=('deptno':20,'name':'RESEARCH') --value-class=pivotal.au.se.deptemp.beans.Department --region=departments;
put --key=30 --value=('deptno':30,'name':'SALES') --value-class=pivotal.au.se.deptemp.beans.Department --region=departments;
put --key=40 --value=('deptno':40,'name':'OPERATIONS') --value-class=pivotal.au.se.deptemp.beans.Department --region=departments;

Emp Data

put --key=7369 --value=('empno':7369,'name':'SMITH','job':'CLERK','deptno':20) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7370 --value=('empno':7370,'name':'APPLES','job':'MANAGER','deptno':10) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7371 --value=('empno':7371,'name':'APICELLA','job':'SALESMAN','deptno':10) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7372 --value=('empno':7372,'name':'LUCIA','job':'PRESIDENT','deptno':30) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7373 --value=('empno':7373,'name':'SIENA','job':'CLERK','deptno':40) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7374 --value=('empno':7374,'name':'LUCAS','job':'SALESMAN','deptno':10) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7375 --value=('empno':7375,'name':'ROB','job':'CLERK','deptno':30) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7376 --value=('empno':7376,'name':'ADRIAN','job':'CLERK','deptno':20) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7377 --value=('empno':7377,'name':'ADAM','job':'CLERK','deptno':20) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7378 --value=('empno':7378,'name':'SALLY','job':'MANAGER','deptno':20) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7379 --value=('empno':7379,'name':'FRANK','job':'CLERK','deptno':10) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7380 --value=('empno':7380,'name':'BLACK','job':'CLERK','deptno':40) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7381 --value=('empno':7381,'name':'BROWN','job':'SALESMAN','deptno':40) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;

Load into GemFire (Assumed JAR for POJO'S exists in class path of GemFireCache Servers)

The script bellows uses GFSH to load the file into the correct region references the correct POJO inside the files created above.

export CUR_DIR=`pwd`

gfsh <<!
connect --locator=localhost[10334];
run --file=$CUR_DIR/dept-data
run --file=$CUR_DIR/emp-data
!

Below is what the Department.java POJO would look like for example.
  
package pivotal.au.se.deptemp.beans;

public class Department
{
private int deptno;
private String name;

public Department()
{
}

public Department(int deptno, String name) {
super();
this.deptno = deptno;
this.name = name;
}

public int getDeptno() {
return deptno;
}

public void setDeptno(int deptno) {
this.deptno = deptno;
}

public String getName() {
return name;
}

public void setName(String name) {
this.name = name;
}

@Override
public String toString() {
return "Department [deptno=" + deptno + ", name=" + name + "]";
}

}
http://feeds.feedburner.com/TheBlasFromPas
Categories: Fusion Middleware

Using HAWQ with PHD service in PCF 1.2

Mon, 2014-07-28 06:01
The following demo shows how to use the PCF 1.2 PHD service with HAWQ by loading data into the PCF PaaS platform.

1. First lets setup our ENV to use the correct version of HADOOP on our local laptop.

export HADOOP_INSTALL=/Users/papicella/vmware/software/hadoop/hadoop-2.0.5-alpha
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home

export PATH=$PATH:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin
export HADOOP_OPTS="$HADOOP_OPTS -Djava.net.preferIPv4Stack=true"

export HADOOP_OPTS="$HADOOP_OPTS  -Djava.awt.headless=true -Djava.security.krb5.realm=-Djava.security.krb5.kdc="

export YARN_OPTS="$YARN_OPTS -Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk -Djava.awt.headless=true"

hadoop version

2. Set the HADOOP_USER_NAME to ensure you have write access to load a file.

export HADOOP_USER_NAME=ucc3a04008db2486

3. Create a file called person.txt with some pipe delimited data , example below.

[Mon Jul 28 21:47:37 papicella@:~/vmware/software/hadoop/cloud-foundry/pcf12/demo ] $ head person.txt
1|person1
2|person2
3|person3
4|person4
5|person5

4. Load the file into the PHD instance running in PCF 1.2. You will need to use the name node / path which is correct for your PHD instance.

[Mon Jul 28 21:51:43 papicella@:~/vmware/software/hadoop/cloud-foundry/pcf12/demo ] $ hadoop fs -put person.txt hdfs://x.x.x.x:8020/user/ucc3a04008db2486/

5. Create a HAWQ table to the file person.txt using PXF as shown below.

CREATE EXTERNAL TABLE person (id int, name text)
LOCATION ('pxf://x.x.x.x:50070/user/ucc3a04008db2486/person.txt?Fragmenter=HdfsDataFragmenter&Accessor=TextFileAccessor&Resolver=TextResolver')
FORMAT 'TEXT' (DELIMITER = '|');

6. Query the table as shown below.



For more information on the PHD service see the link below.

http://docs.pivotal.io/pivotalhd-ds/index.html

http://feeds.feedburner.com/TheBlasFromPas
Categories: Fusion Middleware

Pivotal Cloud Foundry Installed lets create an ORG / USER to get started

Thu, 2014-06-26 18:16
I installed Pivotal Cloud Foundry 1.2 recently and the commands below is what I run using the CLI to quickly create an ORG and a USER to get started with. Below assumes your connected as the ADMIN user to set a new ORG up.

Cloud Foundry CLI Commands as follows

cf api {cloud end point}
cf create-org pivotal
cf create-user pas pas
cf set-org-role pas pivotal OrgManager
cf target -o pivotal
cf create-space development
cf create-space test
cf create-space production
cf set-space-role pas pivotal production SpaceDeveloper
cf set-space-role pas pivotal development SpaceDeveloper
cf set-space-role pas pivotal test SpaceDeveloper
cf login -u pas -p pas -s developmenthttp://feeds.feedburner.com/TheBlasFromPas
Categories: Fusion Middleware

Pivotal GemFireXD*Web, Web based Interface For GemFireXD

Thu, 2014-05-15 04:59
Pivotal GemFire XD bridges GemFire’s proven in-memory intelligence and integrates it with Pivotal HD 2.0 and HAWQ. This enables businesses to make prescriptive decisions in real-time, such as stock trading, fraud detection, intelligence for energy companies, or routing for the telecom industries.

You can read more about how GemFireXD and it's integration with PHD here.

https://www.gopivotal.com/big-data/pivotal-hd

While development team worked on GemFireXD I produced another open source web based tool named GemFireXD*Web. It's available with source code as follows.

https://github.com/papicella/GemFireXD-Web

GemFireXD *Web enables schema management from a web browser with features as follows

  • Create all Schema Objects via Dialogs
  • Generate DDL
  • Run multiple SQL Commands, upload SQL files
  • Browse / Administer Objects
  • Browse / Administer HDFS stores/tables 
  • Browse / Administer Async Event Listeners
  • View data distribution
  • View Members / start parameters


etc…







http://feeds.feedburner.com/TheBlasFromPas
Categories: Fusion Middleware