Skip navigation.

Pas Apicella

Syndicate content
Information on Pivotal -> Product Family - GemFire, SQLFire/GemFireXD, tc Server, RabbitMQ, Pivotal HD, Greenplum DB and SpringPas Apicellahttp://www.blogger.com/profile/09389663166398991762noreply@blogger.comBlogger273125
Updated: 6 min 54 sec ago

Creating a Pivotal GemFireXD Data Source Connection from IntelliJ IDEA 13.x

Wed, 2014-09-10 19:04
In order to create a Pivotal GemFireXD Data Source Connection from IntelliJ 13.x , follow the steps below. You will need to define a GemFireXD driver , prior to creating the Data Source itself.

1. Bring up the Databases panel.

2. Define a GemFireXD Driver as follows


3. Once defined select it by using the following options. Your using the Driver you created at #2 above

+ -> Data Source -> com.pivotal.gemfirexd.jdbc.ClientDriver 

4. Create a Connection as shown below. You would need to having a running GemFireXD cluster at this point in order to connect.



5.  Once connected you can browse objects as shown below.



6. Finally we can run DML/DDL directly from IntelliJ as shown below.


http://feeds.feedburner.com/TheBlasFromPas
Categories: Fusion Middleware

Variable in list with Postgres JDBC and Greenplum

Wed, 2014-09-03 21:59
I previously blogged on how to create a variable JDBC IN list with Oracle. Here is how you would do it with Pivotal Greenplum. Much easier , without having to write a function. In the Greenplum demo below we use the any function combined with string_to_array

http://theblasfrompas.blogspot.com.au/2008/02/variable-in-list-with-oracle-jdbc-and.html

Code as follows
  
import java.sql.*;
import java.sql.DriverManager;

/**
* Created by papicella on 4/09/2014.
*/
public class VariableInListGreenplum
{

public VariableInListGreenplum()
{
}

private Connection getConnection() throws SQLException, ClassNotFoundException
{
Class.forName("org.postgresql.Driver");
Connection conn = null;
conn = DriverManager.getConnection(
"jdbc:postgresql://127.0.0.1:5432/apples","pas", "pas");

return conn;
}

public void run() throws SQLException
{
Connection conn = null;
PreparedStatement stmt = null;
ResultSet rset = null;
String queryInList =
"SELECT DEPTNO, " +
" DNAME, " +
" LOC " +
"FROM scott.DEPT " +
"WHERE DEPTNO = any(string_to_array(?,', ')) ";

try
{
conn = getConnection();
stmt = conn.prepareStatement(queryInList);
stmt.setString(1, "10, 20, 30");
rset = stmt.executeQuery();

while (rset.next())
{
System.out.println("Dept [" + rset.getInt(1) + ", " +
rset.getString(2) + "]");
}
}
catch (Exception e)
{
System.out.println("Exception occurred");
e.printStackTrace();
}
finally
{
if (conn != null)
{
conn.close();
}

if (stmt != null)
{
stmt.close();
}

if (rset != null)
{
rset.close();
}
}
}

public static void main(String[] args) throws Exception
{
VariableInListGreenplum test = new VariableInListGreenplum();
test.run();
}
}
http://feeds.feedburner.com/TheBlasFromPas
Categories: Fusion Middleware

REST with Pivotal GemFire 8.0

Tue, 2014-09-02 21:32
Pivotal GemFire 8.0 now includes REST support. You can read more about it as follows

http://gemfire.docs.pivotal.io/latest/userguide/gemfire_rest/book_intro.html#concept_7628F498DB534A2D8A99748F5DA5DC94

Here is how we set it up and some quick examples showing how it works with some Region data
In the example below I have PDX setup for the cache servers as shown below.
  
<!DOCTYPE cache PUBLIC
"-//GemStone Systems, Inc.//GemFire Declarative Caching 8.0//EN"
"http://www.gemstone.com/dtd/cache8_0.dtd">
<cache>
<pdx read-serialized="true">
<pdx-serializer>
<class-name>com.gemstone.gemfire.pdx.ReflectionBasedAutoSerializer</class-name>
<parameter name="classes">
<string>org\.pivotal\.pas\.beans\..*</string>
</parameter>
</pdx-serializer>
</pdx>

.....

1. Firstly you need to enable the REST on a cache server node as shown below. Basically set gemfire.start-dev-rest-api to TRUE , you could use a gemfire.properties file but here we just pass it to GFSH as part of the server start command.

start server --name=server1 --classpath=$CLASSPATH --server-port=40411 --cache-xml-file=./server1/cache.xml --properties-file=./server1/gemfire.properties --locators=localhost[10334] --dir=server1 --initial-heap=1g --max-heap=1g --J=-Dgemfire.http-service-port=7070 --J=-Dgemfire.http-service-bind-address=localhost --J=-Dgemfire.start-dev-rest-api=true

2. Once started we can quickly ensure we have the REST server up on port 7070 as shown below.

[Wed Sep 03 12:39:18 papicella@:~/ant-demos/gemfire/80/demo ] $ netstat -an | grep 7070
tcp4       0      0  127.0.0.1.7070         *.*                    LISTEN

3. Next test that you can access the REST server. The command below will list all the regions available in the cluster.

[Wed Sep 03 12:52:44 papicella@:~/ant-demos/gemfire/80/demo/rest ] $ curl -i http://localhost:7070/gemfire-api/v1
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Location: http://localhost:7070/gemfire-api/v1
Accept-Charset: big5, big5-hkscs, euc-jp, euc-kr, gb18030, gb2312, gbk, ibm-thai, ibm00858, ibm01140, ibm01141, ibm01142, ibm01143, ibm01144, ibm01145, ibm01146, ibm01147, ibm01148, ibm01149, ibm037, ibm1026, ibm1047, ibm273, ibm277, ibm278, ibm280, ibm284, ibm285, ibm290, ibm297, ibm420, ibm424, ibm437, ibm500, ibm775, ibm850, ibm852, ibm855, ibm857, ibm860, ibm861, ibm862, ibm863, ibm864, ibm865, ibm866, ibm868, ibm869, ibm870, ibm871, ibm918, iso-2022-cn, iso-2022-jp, iso-2022-jp-2, iso-2022-kr, iso-8859-1, iso-8859-13, iso-8859-15, iso-8859-2, iso-8859-3, iso-8859-4, iso-8859-5, iso-8859-6, iso-8859-7, iso-8859-8, iso-8859-9, jis_x0201, jis_x0212-1990, koi8-r, koi8-u, shift_jis, tis-620, us-ascii, utf-16, utf-16be, utf-16le, utf-32, utf-32be, utf-32le, utf-8, windows-1250, windows-1251, windows-1252, windows-1253, windows-1254, windows-1255, windows-1256, windows-1257, windows-1258, windows-31j, x-big5-hkscs-2001, x-big5-solaris, x-compound_text, x-euc-jp-linux, x-euc-tw, x-eucjp-open, x-ibm1006, x-ibm1025, x-ibm1046, x-ibm1097, x-ibm1098, x-ibm1112, x-ibm1122, x-ibm1123, x-ibm1124, x-ibm1364, x-ibm1381, x-ibm1383, x-ibm300, x-ibm33722, x-ibm737, x-ibm833, x-ibm834, x-ibm856, x-ibm874, x-ibm875, x-ibm921, x-ibm922, x-ibm930, x-ibm933, x-ibm935, x-ibm937, x-ibm939, x-ibm942, x-ibm942c, x-ibm943, x-ibm943c, x-ibm948, x-ibm949, x-ibm949c, x-ibm950, x-ibm964, x-ibm970, x-iscii91, x-iso-2022-cn-cns, x-iso-2022-cn-gb, x-iso-8859-11, x-jis0208, x-jisautodetect, x-johab, x-macarabic, x-maccentraleurope, x-maccroatian, x-maccyrillic, x-macdingbat, x-macgreek, x-machebrew, x-maciceland, x-macroman, x-macromania, x-macsymbol, x-macthai, x-macturkish, x-macukraine, x-ms932_0213, x-ms950-hkscs, x-ms950-hkscs-xp, x-mswin-936, x-pck, x-sjis_0213, x-utf-16le-bom, x-utf-32be-bom, x-utf-32le-bom, x-windows-50220, x-windows-50221, x-windows-874, x-windows-949, x-windows-950, x-windows-iso2022jp
Content-Type: application/json
Content-Length: 493
Date: Wed, 03 Sep 2014 02:52:46 GMT

{
  "regions" : [ {
    "name" : "demoRegion",
    "type" : "PARTITION",
    "key-constraint" : null,
    "value-constraint" : null
  }, {
    "name" : "departments",
    "type" : "PARTITION",
    "key-constraint" : null,
    "value-constraint" : null
  }, {
    "name" : "employees",
    "type" : "PARTITION",
    "key-constraint" : null,
    "value-constraint" : null
  }, {
    "name" : "complex",
    "type" : "PARTITION",
    "key-constraint" : null,
    "value-constraint" : null
  } ]

4. We have a couple of regions in this cluster and once again I am using the classic DEPT/EMP regions here. Some simple REST command belows on the "/departments" region

View all DEPARTMENT region entries

[Wed Sep 03 12:53:38 papicella@:~/ant-demos/gemfire/80/demo/rest ] $ curl -i http://localhost:7070/gemfire-api/v1/departments
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Location: http://localhost:7070/gemfire-api/v1/departments/20,10,30,40
Content-Type: application/json
Content-Length: 225
Date: Wed, 03 Sep 2014 02:53:40 GMT

{
  "departments" : [ {
    "deptno" : 20,
    "name" : "RESEARCH"
  }, {
    "deptno" : 10,
    "name" : "ACCOUNTING"
  }, {
    "deptno" : 30,
    "name" : "SALES"
  }, {
    "deptno" : 40,
    "name" : "OPERATIONS"
  } ]
}

VIEW a single region entry by KEY

[Wed Sep 03 12:55:34 papicella@:~/ant-demos/gemfire/80/demo/rest ] $ curl -i http://localhost:7070/gemfire-api/v1/departments/10
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Location: http://localhost:7070/gemfire-api/v1/departments/10
Content-Type: application/json
Content-Length: 44
Date: Wed, 03 Sep 2014 02:55:36 GMT

{
  "deptno" : 10,
  "name" : "ACCOUNTING"
}

VIEW multiple entries by KEY

[Wed Sep 03 12:56:25 papicella@:~/ant-demos/gemfire/80/demo/rest ] $ curl -i http://localhost:7070/gemfire-api/v1/departments/10,30
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Location: http://localhost:7070/gemfire-api/v1/departments/10,30
Content-Type: application/json
Content-Length: 123
Date: Wed, 03 Sep 2014 02:56:28 GMT

{
  "departments" : [ {
    "deptno" : 10,
    "name" : "ACCOUNTING"
  }, {
    "deptno" : 30,
    "name" : "SALES"
  } ]
}

5. We can even use the Spring REST shell as shown below.

Obtain rest-shell using the link below.

https://github.com/spring-projects/rest-shell

  
[Wed Sep 03 13:06:22 papicella@:~ ] $ rest-shell

___ ___ __ _____ __ _ _ _ _ __
| _ \ __/' _/_ _/' _/| || | / / | \ \
| v / _|`._`. | | `._`.| >< | / / / > >
|_|_\___|___/ |_| |___/|_||_| |_/_/ /_/
1.2.1.RELEASE

Welcome to the REST shell. For assistance hit TAB or type "help".
http://localhost:8080:> baseUri http://localhost:7070/
Base URI set to 'http://localhost:7070'
http://localhost:7070:> follow gemfire-api
http://localhost:7070/gemfire-api:> follow v1
http://localhost:7070/gemfire-api/v1:> follow departments
http://localhost:7070/gemfire-api/v1/departments:> get 20
> GET http://localhost:7070/gemfire-api/v1/departments/20

< 200 OK
< Server: Apache-Coyote/1.1
< Content-Location: http://localhost:7070/gemfire-api/v1/departments/20
< Content-Type: application/json
< Content-Length: 42
< Date: Wed, 03 Sep 2014 03:07:17 GMT
<
{
"deptno" : 20,
"name" : "RESEARCH"
}
http://localhost:7070/gemfire-api/v1/departments:>

6. Open a browser and enter the following URL to browse the Swagger-enabled REST APIs:

http://localhost:7070/gemfire-api/docs/index.html



7. Perform an operation as shown below.




http://feeds.feedburner.com/TheBlasFromPas
Categories: Fusion Middleware

Dept/Emp POJO's with sample data for Pivotal GemFire

Tue, 2014-08-12 21:57
I constantly blog about using DEPARTMENT/EMPLOYEE POJO'S with sample data. Here is how to create a file with data to load into GemFire to give you that sample set.

Note: You would need to create POJO'S for Department/Empployee objects that have getter/setter for the attributes mentioned below.

Dept Data

put --key=10 --value=('deptno':10,'name':'ACCOUNTING') --value-class=pivotal.au.se.deptemp.beans.Department --region=departments;
put --key=20 --value=('deptno':20,'name':'RESEARCH') --value-class=pivotal.au.se.deptemp.beans.Department --region=departments;
put --key=30 --value=('deptno':30,'name':'SALES') --value-class=pivotal.au.se.deptemp.beans.Department --region=departments;
put --key=40 --value=('deptno':40,'name':'OPERATIONS') --value-class=pivotal.au.se.deptemp.beans.Department --region=departments;

Emp Data

put --key=7369 --value=('empno':7369,'name':'SMITH','job':'CLERK','deptno':20) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7370 --value=('empno':7370,'name':'APPLES','job':'MANAGER','deptno':10) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7371 --value=('empno':7371,'name':'APICELLA','job':'SALESMAN','deptno':10) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7372 --value=('empno':7372,'name':'LUCIA','job':'PRESIDENT','deptno':30) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7373 --value=('empno':7373,'name':'SIENA','job':'CLERK','deptno':40) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7374 --value=('empno':7374,'name':'LUCAS','job':'SALESMAN','deptno':10) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7375 --value=('empno':7375,'name':'ROB','job':'CLERK','deptno':30) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7376 --value=('empno':7376,'name':'ADRIAN','job':'CLERK','deptno':20) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7377 --value=('empno':7377,'name':'ADAM','job':'CLERK','deptno':20) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7378 --value=('empno':7378,'name':'SALLY','job':'MANAGER','deptno':20) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7379 --value=('empno':7379,'name':'FRANK','job':'CLERK','deptno':10) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7380 --value=('empno':7380,'name':'BLACK','job':'CLERK','deptno':40) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;
put --key=7381 --value=('empno':7381,'name':'BROWN','job':'SALESMAN','deptno':40) --value-class=pivotal.au.se.deptemp.beans.Employee --region=employees;

Load into GemFire (Assumed JAR for POJO'S exists in class path of GemFireCache Servers)

The script bellows uses GFSH to load the file into the correct region references the correct POJO inside the files created above.

export CUR_DIR=`pwd`

gfsh <<!
connect --locator=localhost[10334];
run --file=$CUR_DIR/dept-data
run --file=$CUR_DIR/emp-data
!

Below is what the Department.java POJO would look like for example.
  
package pivotal.au.se.deptemp.beans;

public class Department
{
private int deptno;
private String name;

public Department()
{
}

public Department(int deptno, String name) {
super();
this.deptno = deptno;
this.name = name;
}

public int getDeptno() {
return deptno;
}

public void setDeptno(int deptno) {
this.deptno = deptno;
}

public String getName() {
return name;
}

public void setName(String name) {
this.name = name;
}

@Override
public String toString() {
return "Department [deptno=" + deptno + ", name=" + name + "]";
}

}
http://feeds.feedburner.com/TheBlasFromPas
Categories: Fusion Middleware

Using HAWQ with PHD service in PCF 1.2

Mon, 2014-07-28 06:01
The following demo shows how to use the PCF 1.2 PHD service with HAWQ by loading data into the PCF PaaS platform.

1. First lets setup our ENV to use the correct version of HADOOP on our local laptop.

export HADOOP_INSTALL=/Users/papicella/vmware/software/hadoop/hadoop-2.0.5-alpha
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home

export PATH=$PATH:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin
export HADOOP_OPTS="$HADOOP_OPTS -Djava.net.preferIPv4Stack=true"

export HADOOP_OPTS="$HADOOP_OPTS  -Djava.awt.headless=true -Djava.security.krb5.realm=-Djava.security.krb5.kdc="

export YARN_OPTS="$YARN_OPTS -Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk -Djava.awt.headless=true"

hadoop version

2. Set the HADOOP_USER_NAME to ensure you have write access to load a file.

export HADOOP_USER_NAME=ucc3a04008db2486

3. Create a file called person.txt with some pipe delimited data , example below.

[Mon Jul 28 21:47:37 papicella@:~/vmware/software/hadoop/cloud-foundry/pcf12/demo ] $ head person.txt
1|person1
2|person2
3|person3
4|person4
5|person5

4. Load the file into the PHD instance running in PCF 1.2. You will need to use the name node / path which is correct for your PHD instance.

[Mon Jul 28 21:51:43 papicella@:~/vmware/software/hadoop/cloud-foundry/pcf12/demo ] $ hadoop fs -put person.txt hdfs://x.x.x.x:8020/user/ucc3a04008db2486/

5. Create a HAWQ table to the file person.txt using PXF as shown below.

CREATE EXTERNAL TABLE person (id int, name text)
LOCATION ('pxf://x.x.x.x:50070/user/ucc3a04008db2486/person.txt?Fragmenter=HdfsDataFragmenter&Accessor=TextFileAccessor&Resolver=TextResolver')
FORMAT 'TEXT' (DELIMITER = '|');

6. Query the table as shown below.



For more information on the PHD service see the link below.

http://docs.pivotal.io/pivotalhd-ds/index.html

http://feeds.feedburner.com/TheBlasFromPas
Categories: Fusion Middleware

Pivotal Cloud Foundry Installed lets create an ORG / USER to get started

Thu, 2014-06-26 18:16
I installed Pivotal Cloud Foundry 1.2 recently and the commands below is what I run using the CLI to quickly create an ORG and a USER to get started with. Below assumes your connected as the ADMIN user to set a new ORG up.

Cloud Foundry CLI Commands as follows

cf api {cloud end point}
cf create-org pivotal
cf create-user pas pas
cf set-org-role pas pivotal OrgManager
cf target -o pivotal
cf create-space development
cf create-space test
cf create-space production
cf set-space-role pas pivotal production SpaceDeveloper
cf set-space-role pas pivotal development SpaceDeveloper
cf set-space-role pas pivotal test SpaceDeveloper
cf login -u pas -p pas -s developmenthttp://feeds.feedburner.com/TheBlasFromPas
Categories: Fusion Middleware