Collabnet SVN pre-commit hooks for Folders & Files

In Collabnet Subversion, if you have to put pre-commit hooks for certain folders like build, dist, bin, obj folders etc, you can use the following script:

Include the following at the end in File : /opt/csvn/data/repositories/pre-commit.sh

# Make sure that the log message contains some text.

SVNLOOK=/opt/csvn/bin/svnlook

${SVNLOOK} log -t "$TXN" "$REPOS" | /bin/grep '[[:print:]]\{20\}' > /dev/null || NO_COMMIT_MESSAGE="stop"

if [[ -n "${NO_COMMIT_MESSAGE}" ]]

then

  MORE_THAN_ALLOWED_CHARS=1

else

  #--- Exclude a specific folder for such 

  XTEST=`echo "${REPOS}" | /bin/awk '/commonlibs/ { print 1; exit}' 2>&1`

  if [[ "${XTEST}" != "1" ]]

  then

    NO_COMMIT_MESSAGE=`${SVNLOOK} changed "$REPOS" | tr A-Z a-z | /bin/awk '/\/build\// || /\/dist\// || /\.jar/ { print "stop"; exit } ' 2>&1 `

  fi

fi

if [[ "${NO_COMMIT_MESSAGE}" == "stop" ]]
 then
  echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!" 1>&2
  echo 1>&2
  if [[ ${MORE_THAN_ALLOWED_CHARS} -eq 1 ]]
  then
    echo "Please provide detailed description (min 20 char) about the changes you made in the commit message." 1>&2
  else
    echo "Please exclude folders like 'build' or 'dist' and files like '*.jar' in commit." 1>&2
  fi
  echo 1>&2
  echo "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!" 1>&2
  exit 1
 else
  exit 0
fi

=====================================================================
If you have forgotten the admin password,

you can shut down the csvn service and then go to the file:
csvn-production-hsqldb.script

and change the line referencing admin user to :

INSERT INTO USER VALUES(1,11,'$2a$13$5OuIEVEvKLUHTc/kl5kCROMgvqrlgHsdpoZSWqnaKXR6dMfSHcloO',TRUE,'admin','sunambiar@abc.in','admin user','Super Administrator')

Configuring CollabNet SubVersion

Configuring Collabnet SVN :

https://www.open.collab.net/nonav/downloads/subversion/readme/csvne_linux32linux64.html

CollabNet Subversion Edge - Linux 32/64-bit
Version 2.3.0
Release Date: March 2012

Contents

   1. Overview
   2. Platform and configuration
   3. Requirements
   4. Installation notes
   5. Updates
   6. Documentation
   7. Known issues
   8. Support for CollabNet Subversion Edge
   9. About Subversion and CollabNet
   
1. Overview

   CollabNet Subversion Edge includes everything you need to install, manage
   and operate a Subversion server.  It includes all of the Subversion and
   Apache binaries needed to run a Subversion server.  It also integrates the
   popular ViewVC repository browsing tool so that users can view repository
   history from a web browser.

   CollabNet Subversion Edge also includes a powerful web-based management
   console that makes it easy to configure and manage your Apache server and
   Subversion repositories.  You can easily setup the server to use SSL
   connections and even connect the server to a corporate LDAP repository,
   including Microsoft Active Directory.

   CollabNet Subversion Edge only supports serving Subversion repositories via
   Apache httpd and it only supports the Subversion fsfs repository format.
  
2. Platform and configuration

   Product: CollabNet Subversion Edge
   License: GNU Affero General Public License 3.0 (AGPLv3)
   Certified platforms: Red Hat Enterprise Linux 6.x and 5.x 
                        CentOS 5.x
                        SuSE Linux Enterprise 11.x
   
   NOTE: These are the platforms we formally certify when testing.  CollabNet
   Subversion Edge is known to work on virtually all Linux distributions and is
   informally tested on others such as Ubuntu and Fedora.
   
   There are separate downloads for 32-bit and 64-bit Linux.  Download the 
   appropriate version for your distribution and kernel.
   
3. Requirements
   
   * Java 1.6 JRE/JDK must be installed. We recommend the JRE provided by
     Sun/Oracle.  When testing on 64-bit Linux we have used the 64-bit JVM.
   
   * Python 2.4 to 2.6 must be installed.
 
4. Installation Notes

   IMPORTANT: Do not untar CollabNet Subversion Edge using root or sudo.  This
   will cause the UID/GID on the files to carry the values from our build
   system instead of being reset.
   
   1. Set the JAVA_HOME environment variable, and point it to your Java 6 JRE
      home.  For example:

      export JAVA_HOME=/usr/java/default
   
      Test the variable:
   
      $ $JAVA_HOME/bin/java -version
      java version "1.6.0_20"
      Java(TM) SE Runtime Environment (build 1.6.0_20-b02)
      Java HotSpot(TM) Client VM (build 16.3-b01, mixed mode, sharing)
   
   2. Switch to the folder where you want to install CollabNet Subversion
      Edge.  You must have write permissions to this folder.
      
      $ cd /opt
      
   3. Untar the file you downloaded from CollabNet.
   
      $ tar zxf CollabNetSubversionEdge-x.y.z_linux-x86.tar.gz
      
      This will create a folder named "csvn" in the current directory. You can
      rename this folder if desired.
      
   4. Optional. Install the application so that it will start automatically
      when the server restarts.  This command generally requires root/sudo to
      execute.
      
      $ cd csvn
      $ sudo -E bin/csvn install
      
      In addition to configuring your system so that the server is started
      with the system, it will also write the current JAVA_HOME and the
      current username in to the file data/conf/csvn.conf.  You can edit this
      file if needed as it controls the startup settings for the application.
      By setting the JAVA_HOME and RUN_AS_USER variables in this file, it
      ensures they are set correctly when the application is run.
      
   5. Optional. Configure proxy settings.  CollabNet Subversion Edge need
      access to the internet to check for and install updates.  If you need to
      go through a proxy to access the internet, then you can configure the
      proxy by editing the data/conf/csvn.conf file which was created by the
      previous step.  Uncomment and edit the HTTP_PROXY variable to configure
      your proxy server.
      
   6. Start the server.  Be sure that you are logged in as your own userid and
      not running as root.
      
      $ bin/csvn start
      
      This will take a few minutes and the script will loop until it sees that
      the server is running.  If the server does not start, then try starting
      the server with this command:
      
      $ bin/csvn console
      
      This will start the server but output the initial startup messages to
      the console.

   You must login to the CollabNet Subversion Edge browser-based management
   console and configure the Apache server before it can be run for the first
   time.  The UI of the management console writes the needed Apache
   configuration files based on the information you provide.

   The default administrator login is:

   Address: http://localhost:3343/csvn
   Username: admin
   Password: admin
   
   Subversion Edge also starts an SSL-protected version using a self-signed SSL
   certificate.  You can access the SSL version on this URL:
   
   Address: https://localhost:4434/csvn
 
   You can force users to use SSL from the Server configuration.  This will cause
   attempts to access the site via plain HTTP on port 3343 to be redirected to the
   secure port on 4434.

5. Updates

   CollabNet Subversion Edge includes a built-in mechanism for discovering and 
   installing updates.  You must use this facility to install updates.  Do not
   download and run a new version of the application installer.

   The update mechanism will require you to restart the servers at the end of
   the process, but it will do it for you.

6. Documentation
   
   Documentation for CollabNet Subversion Edge is available here:

   http://help.collab.net/
   
   Context-sensitive help is also linked to this location from within the
   application.
  

7. Known issues

   - For the latest FAQ, visit the project home page here:
   
     https://ctf.open.collab.net/sf/projects/svnedge

   - If you try to access an existing BDB (Berkeley DB) based repository
     through CollabNet Subversion Edge, then you will receive an alert "Failed
     to load module for FS type 'bdb'." This is because CollabNet Subversion
     Edge does not support BDB.  CollabNet recommends FSFS over BDB for ease
     of maintenance and supportability.
     
    - You can access the application from localhost, but not other computers
    on your LAN.  This usually means that you need to configure Firewall rules
    for you server to open the firewall for port 3343:
    
 edit /etc/sysconfig/iptables
 edit /etc/sysconfig/ip6tables
  
 Add the following line to the above files, it line be will the line before
 the last 2 lines.
  
 -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 3343 -j ACCEPT

 /etc/init.d/iptables restart
 /etc/init.d/ip6tables restart

8. Support for CollabNet Subversion Edge

   Ask questions and get assistance with using CollabNet Subversion Edge via
   the community forums on openCollabNet.  The forum for CollabNet Subversion
   Edge questions is available here:

   http://subversion.open.collab.net/ds/viewForumSummary.do?dsForumId=3

   Find out about CollabNet Technical Support at 
   http://www.open.collab.net/support/
   
   Information about CollabNet Training, Consulting, and Migration
   services is at http://www.open.collab.net/training/
   
   Join openCollabNet for community support: http://open.collab.net

9. About Subversion and CollabNet

   CollabNet launched the Subversion project in 2000 in response to the demand 
   for an open standard for Web-based software configuration management that 
   could support distributed development. CollabNet continues to be strongly 
   involved with the Subversion project and offers CollabNet Subversion
   Support, Training, and Consulting services. 
   
   CollabNet also provides the most widely used collaborative development 
   environment in the world. More than 1,400,000 developers and IT projects 
   managers collaborate online through CollabNet. The company is transforming 
   the way software is developed by enabling organizations to leverage global 
   development talents to deliver better products and innovate faster. 
   
   Visit CollabNet at http://www.collab.net for more information.
   
   Subversion is a registered trademark of the Apache Software Foundation.
   http://subversion.apache.org/

snFilterWrapper for Selective Filtering

snFilterWrapper
(v24.0 dtd. 06-Jun-2015)

I had been posted sometime back with one of my colleagues about a use case wherein a single application might require to have web-pages that are accessed by different sections of users.  While not much time could be devoted earlier, it couldn't be left to the wild either.  Developing on it further, one could even think of the scenario of having a single application being accessed on the intranet and internet simultaneously.  As to the question why at all that application should be single, the answer would be that the code, functionality as well as the database is common to the intranet users and the internet users.  There might however be only some web-pages which are not to be accessed by few identified groups.  

Let us take the example of a web-application like the Complaint / Feedback system which has an engine that is to primarily serve

• The internet users who might not need to login at all (with any user id or password) into the system.  
• Another section of users who are from the department designated to handling the complaint/feedback and who are authorized to reply to the feedback could be intranet users of the organization. 
• Yet another group of users could be the like of a Call Centre users who accept and record the feedback and have separate authentication credentials. 
• Another group are the secondary users of the extranet (say the organization's business partners) who could also be required to access and handle the feedbacks.
• There could still be another group which are application users who interact not through any GUI, but through servlets directly from other applications through encrypted/non-encrypted mode. (See snInduceParamsScreenApprover filter for encrypted application level communication).

Sat-on therefore, last few days to develop this feature through a FilterWrapper wherein more than one standard filters could be wrapped - at the same time providing exclusion and inclusion options. The specifications of the implementation is given below for any earnest implementer. 

A simple sample could be the CAS implementation as given below.  In the former part, the traditional methodology is given and in the latter part, the modified methodology is shown. 

---

Traditional Methodology for JASIG-CAS example:-

---

  <filter>

      <filter-name>CAS Authentication Filter</filter-name>

      <filter-class>org.jasig.cas.client.authentication.AuthenticationFilter</filter-class>

      <init-param>

          <param-name>casServerLoginUrl</param-name>

          <param-value>https://spandan.indianoil.co.in/JaanasiMaamVaa/login</param-value>

      </init-param>

      <init-param>

          <param-name>serverName</param-name>

          <param-value>http://localhost:8084</param-value>

      </init-param>

  </filter>

  <filter>

      <filter-name>CAS Validation Filter</filter-name>

      <filter-class>org.jasig.cas.client.validation.Cas10TicketValidationFilter</filter-class>

      <init-param>

          <param-name>casServerUrlPrefix</param-name>

          <param-value>https://spandan.indianoil.co.in/JaanasiMaamVaa/</param-value>

      </init-param>

      <init-param>

          <param-name>serverName</param-name>

          <param-value>http://localhost:8084</param-value>

      </init-param>

      <init-param>

          <param-name>redirectAfterValidation</param-name>

          <param-value>true</param-value>

      </init-param>

  </filter>

  <filter>

      <filter-name>CAS HttpServletRequest Wrapper Filter</filter-name>

      <filter-class>org.jasig.cas.client.util.HttpServletRequestWrapperFilter</filter-class>

  </filter>

  <filter-mapping>

      <filter-name>CAS Authentication Filter</filter-name>

      <url-pattern>/*</url-pattern>

  </filter-mapping>

  <filter-mapping>

      <filter-name>CAS Validation Filter</filter-name>

      <url-pattern>/*</url-pattern>

  </filter-mapping>

  <filter-mapping>

      <filter-name>CAS HttpServletRequest Wrapper Filter</filter-name>

      <url-pattern>/*</url-pattern>

  </filter-mapping>

---

Modified Methodology for the same JASIG-CAS given above with Exclusion & Inclusion Options example:-

---

<filter>

    <filter-name>CAS Authentication Combined-Filter</filter-name>

    <filter-class>org.sun.raibmanus.filters.snFilterWrapper</filter-class>

    <init-param>

      <param-name>ENABLED</param-name>

      <param-value>1</param-value>

    </init-param>

    <init-param>

      <param-name>FilterClassName-1</param-name>

      <param-value>org.jasig.cas.client.authentication.AuthenticationFilter</param-value>

    </init-param>

    <init-param>

      <param-name>FilterClassName-2</param-name>

      <param-value>org.jasig.cas.client.validation.Cas10TicketValidationFilter</param-value>

    </init-param>

    <init-param>

      <param-name>FilterClassName-3</param-name>

      <param-value>org.jasig.cas.client.util.HttpServletRequestWrapperFilter</param-value>

    </init-param>   

    <init-param>

      <param-name>casServerLoginUrl</param-name>

      <param-value>https://spandan.indianoil.co.in/JaanasiMaamVaa/login</param-value>

    </init-param>

    <init-param>

      <param-name>casServerUrlPrefix</param-name>

      <param-value>https://spandan.indianoil.co.in/JaanasiMaamVaa/</param-value>

    </init-param>

    <init-param>

      <param-name>serverName</param-name>

      <param-value>http://localhost:8084</param-value>

    </init-param>     

    <init-param>

      <param-name>exclude_url-2</param-name>

      <param-value>/folder/index_[3-5].jsp</param-value>

    </init-param>  

    <init-param>

      <param-name>exclude_url-ABC</param-name>

      <param-value>index.jsp</param-value>

    </init-param>   

    <init-param>

      <param-name>exclude_url-1</param-name>

      <param-value>index_2.jsp</param-value>

    </init-param>      

    <init-param>

      <param-name>exclude_url-4</param-name>

      <param-value>([\w||\-||/])+\.([gG][iI][fF]|[pP][dD][fF]|[pP][nN][gG]|[jJ][pP][gG]|[jJ][pP][eE][gG]|[bB][mM][pP]|[jJ][sS]|[cC][sS][sS]|[sS][wW][fF]|[iI][cC][oO])</param-value>

    </init-param>        

    <init-param>

      <param-name>include_url-2</param-name>

      <param-value>/folder/index_4.jsp</param-value>

    </init-param>  

  </filter>

  <filter-mapping>

    <filter-name>CAS Authentication Combined-Filter</filter-name>

    <url-pattern>/*</url-pattern>

  </filter-mapping>

---

As can be seen above, the standard filters have been passed as parameter to the snFilterWrapper for its consumption.  The order of the chain is governed by the param-name which should necessarily begin with the characters 'FilterClassName'.  By convention it could be named 

• FilterClassName00, FilterClassName01, FilterClassName02 etc.  

 or

• FilterClassName_1, FilterClassName_2, FilterClassName_3 etc. 

or even 

• FilterClassName_A, FilterClassName_B, FilterClassName_C etc.  

However, it should be noted that the name is case-insensitive so that 'FilterClassName_A' and 'filterclassName_a'  are identified to the same.

The regular expressions under the param-names exclude-url and include-url with appropriate suffixes shall govern the permeability of the filter.  They are optional and are needed only if the use-case mentioned in the beginning have to be implemented.

A new parameter call 'Enabled' has been introduced with version 25.2 (dtd. 28.09.2015).  This can come in handy especially along with JASIG-CAS where the CAS Filter could be disabled easily while in Developer mode.  0 or FALSE value indicates that the filter should be switched off.  1 or TRUE shall indicate that the filter be switched On.   The default mode is ON.

Ideally, the above could be combined with snInduceParamsScreenApprover  to improve the robustness of the application in terms of Application Security.

---

snLibrary - utils

snLibrary - utils

There are a few utilities that can be used frequently in our codes. These have been packaged into utils package. Some other useful ones are also available under snUploader package as well.

Class: utils.snUtils:-
 num2words() - Convert a numeric (long) value to words in Indian format
 dmy2ymd() - Convert a String in Date format of YYYY-MM-DD into DD-MM-YYYY format
 ymd2dmy() - Convert a String in Date format of DD-MM-YYYY into YYYY-MM-DD format.
 getEncodeUTF8() - URLEncode  the String parameter.

Class: config.snDBFuncLib:-
  getSQLDate() - Gives the current date in YYYY-MM-DD (ie SQL standard format).
  getSQLDateTime() - Gives the current date in YYYY-MM-DD HH24:MI:SS  (ie SQL standard format).
 
Class: utils.snUtilsConstants:-
  getDateTimeNow() - sets Date and Time in two public variables namely : DATE_NOW and TIME_NOW.  The format for Date is DD-MM-YYYY Gives the current date in YYYY.MM.DD and that for time is HH.MM.SS.  (May be converted to required format using dmy2ymd()).

---

snLibrary - XML file Upload into DB

snLibrary - XML file Upload into DB

I have faced many situations where I wanted to upload an XML file straight into a table.  Have you too?

Of course, only select XML tags should be picked up from the XML.  In a separate package, called utils, along with some other utility classes, snLibrary contains few functions for achieving this uploading.  It can be used not only in a web-application, but also in a normal java application that could pick up the XML file from the file-system and upload the contents in the specified table in the database.

The method and its usage is given below.
Method:- utils.snUploader :: snReadXMLintoDBwithInsert()
// This method has been used widely for Uploading the XML file received from SAP after calling
//      select RFCs (Remote Function Calls) of SAP.
// ...
// ... Some java code goes here..
// ...

  snUtilsConstants snUC = new snUtilsConstants();
  // snUtilsConstants contains few Constant values that are defined so as to be used as Data Type
  //      in the following method.

  final String[] XML_TAGS = {"VENDCODE","TITLE","VENDNAME","DISTRICT","STATE","AMOUNT","POSTING_DATE" };
  // Here we have defined the XML Tags as it appears in the XML file.

  final String[] UPLOAD_COLS = {"CARRIER_ID","TITLE","CARRIER_NAME","DISTRICT","STATE","AMOUNT","ACCNG_DATE" };
//  Here we have defined the corresponding Columns of the Table in our database.

  final Object[] UPLOAD_COL_TYPES = {0L, null, null, null, null, 0.00d,
                                            snUC.DT_TYPE_DMYY };  // null corresponds to String Type
  // Here we define the data type of each Column.   While null or simply "" could represent a
  //    String datatype, other datatypes are defined in the snUtilsConstants class.
  //    snUtilsConstants.DT_TYPE_DMYY denotes Date datatype given in the format dd_mm_yyyy

  snUploader snUpd = new snUploader();

  // dbPropXMLFileName is an XML file that contains the Connection Parameters
  //                                            for connecting to the database server.
  //  The parameters are : Driver, JDBCurl, login, pass
  Connection con = snUpd.createConnection(dbPropXMLFileName);

  int insRows = snUpd.snReadXMLintoDBwithInsert(inputXMLFileName, con, dbTablename,
                        rootSearchItemId, XML_TAGS, UPLOAD_COLS, UPLOAD_COL_TYPES);
  //  or to use with the Truncate Option,
  boolean no_truncate = false;
  int insRows = snUpd.snReadXMLintoDBwithInsert(inputXMLFileName, con, no_truncate,
       dbTablename, rootSearchItemId, XML_TAGS, UPLOAD_COLS, UPLOAD_COL_TYPES);
  con.close();
  // ...
  // ... Other part of the code
  // ...

Few other functions/methods are also available for example,
   snReadXMLintoDBwithInsertUpdate() will try an insert for each record and if it fails, will try an update on the same.
  snReadXMLintoDBwithUpdateInsert() will try an update first for each record and if it fails, will try an insert on the same.
  snReadXMLintoDBwithUpdate() will only try to update if similar record exists.

---

snLibrary - FileUploader

snLibrary - FileUploader

(v14.0 dtd. 22-Feb-2012)

When we create forms where we need to upload files to the server from the client side, the following input tag is used.

<html> <head> <title>File Uploading Form</title> </head>

   

 <body> 

   

<h3>File Upload:</h3> Select a file to upload: <br /> 

   

<form action="UploadServlet" method="post" enctype="multipart/form-data">

 <INPUT type="file" name="myfileforUpload" size="50" />

 <br />

 <input type="submit" value="Upload File" />

 </form> 

   

</body> </html>

In the servlet / jsp code, the following can be used.
File:- UploadServlet.java
import servlets.uploader.snUploader;
  
   snUploader snUpld = new snUploader();
             DataInputStream dataStream = new DataInputStream(snUpld.snGetUploadedFileItem(request, 
                                                                            "myfileforUpload").getInputStream());
   //     - or simply -
   DataInputStream dataStream = snUpld.snGetUploadedFileDataStream(request,  
                                                                    "myfileforUpload");
   BufferedReader buffReader = new BufferedReader(new InputStreamReader(dataStream, 
                                                                     "UTF-8"));
   for (String line; (line = buffReader.readLine()) != null;) {
        out.println(line + "<br/>\n"); 
        /// Some processing of the input file
   }

---

snLibrary - with LDAP / Active-Directory

snLibrary - with LDAP / Active-Directory

Probably it was sometime in 2011 that I started some work the Active Directory - though I had earlier conducted some experiments with LDAP of Lotus Domino for Java application interfacing.

I will now narrate some useful methods that you would like to implement for connecting to any LDAP (even Active Directory). 

Create a new folder under c:/snConfig by name LDAP.  Make an entry in the snConfigFile as follows.

File:- c:/snConfig/snConfigMyApplication.ini

LDAP Interface Configuration File=c:/snConfig/LDAP/snLDAPCfg.ini

Create the specified LDAP-InterfaceCfgFile c:/snConfig/LDAP/snLDAPCfg.ini as follows.

File:- c:/snConfig/LDAP/snLDAPCfg.ini 

No. of LDAP Servers=2

0-Initial Context Factory=com.sun.jndi.ldap.LdapCtxFactory  

0-Base Name=DC=DS,DC=INDIANOIL,DC=IN   

0-Search Filter=(&(objectClass=user)(userPrincipalName=*#{USER}*))  

#   Note:-  #{USER} keyword would be replaced by the UserName passed while calling the appropriate method.  

0-Return Attributes=sn, name, givenName, title, description, displayName, mail, designation, department

0-Provider URL=ldap://dcmkho1.ds.indianoil.in:389 

#0-Provider URL=ldap://dcmkho1.ds.indianoil.in:636  

0-Security Authentication=simple  

0-Domain=DS.INDIANOIL.IN 

#0-Service User=00023569

0-Service User=serviceuser  

0-Service Passwd=**********   

#0-Security Protocol=ssl  

0-Id ShortName=sn  

0-Id UserName=uid  

0-Id DisplayName=displayName  

0-Id Designation=title  

0-Id Department=department  

0-Id Mail=mail  

1-Initial Context Factory=com.sun.jndi.ldap.LdapCtxFactory  

1-Base Name=DC=DS,DC=INDIANOIL,DC=IN   

1-Search Filter=(&(objectClass=user)(userPrincipalName=*#{USER}*))  

#   Note:-  #{USER} keyword would be replaced by the UserName passed while calling the appropriate method.  

1-Return Attributes=sn, name, givenName, title, description, displayName, mail, designation, department

1-Provider URL=ldap://dcmkho2.ds.indianoil.in:389 

#1-Provider URL=ldap://dcmkho2.ds.indianoil.in:636  

1-Security Authentication=simple  

1-Domain=DS.INDIANOIL.IN 

#1-Service User=00023569

1-Service User=serviceuser  

1-Service Passwd=**********   

#1-Security Protocol=ssl  

1-Id ShortName=sn  

1-Id UserName=uid  

1-Id DisplayName=displayName  

1-Id Designation=title  

1-Id Department=department  

1-Id Mail=mail  

The configuration given above assumes that there are two LDAP servers namely dcmkho1.ds.indianoil.in and dcmkho2.ds.indianoil.in (In this case, both happen to be Active Directory setup).  The rule is that the LDAP server is identified by the prefix number given viz. 0, 1 etc. The maximum number of servers is also mentioned herein so that it will not consider more entries beyond that.

Now to go into the Java Code, .....

File:- myLDAPexample.java

import java.util.concurrent.ConcurrentHashMap;

import connect.ldap.snLDAPConnector;

// ...

// ... Some part of the code goes here ...

// ...

    ConcurrentHashMap hT = new ConcurrentHashMap();

    

     int id = 1;   // Refer to the LDAP configuration named 1

    snLDAPConnector snLDAP = new snLDAPConnector(id); 
    /// id is say 0 or 1 or 2 as defined in the Configuration file

    hT = snLDAP.fetchUserAttributes("00023569");

    String shortname = (String)hT.get("sn"));

    String name = (String)hT.get("name"));

    String designation = (String)hT.get(snLDAP.getID_Designation(id)));

    String displayName = (String)hT.get(snLDAP.getID_DisplayName(id))); 

// ...

// ... Rest part of the code ....

// ...

Quite easy !! Isn't it?

---

snLibrary - Encryption & Decryption - Inter-Application Communication

snLibrary - Encryption & Decryption

(v10.2 dtd. 07-Dec-2010)

Until the advent of CAS (Central Authentication Service), authentication used to be done at the application level by prompting the user for a User-ID and a Password.  Those parameters are then compared with that stored in the database inorder to assertain whether or  not to allow the user to proceed with operating the application.

There are two methods of verification of password. For example.
Eg. 1:-
a. Obtain the password from the user
b. Verify whether the password entered is the same as that in the database table by comparing the obtained password with the decrypted password obtained from the table (assuming that the password has been stored in the encrypted form in the table).
c.  If the password entered and the password decrypted are the same, allow to proceed with the application; otherwise not.


Eg. 2:-
a. Obtain the password from the user.
b. Encrypt the password obtained from the user.
c. Verify whether the encrypted password entered is the same as that stored in the database table, by comparing both of them.
d. If the passwords match, allow to proceed with the application; otherwise not.

In my opinion, the second is the better method as it is more secure.  Further security can be ascertained by the following form of the SQL.
query = "SELECT userid, userpswd FROM USER_TABLE where userpswd = ?  and userid = ?";
pS = con.prepareStatement(query);

pS.setString(1, user_id);
pS.setString(2, snPassword.getInstance().encrypt(password));
pS.executeQuery();

Having obtained the result set after executing the SQL, once again verify the userid and userpswd with that obtained.
       if (userid.equals(rs.getString(1)) && userpswd.equals(rs.getString(2))) {
            // proceed further on with the application 
       } else {
          // throw Error in User Id / Password 
       }
It is advised to never use any of the following SQLs for application security reasons as they are prone to bugs/SQL injections:
query = "SELECT * FROM USER_TABLE WHERE userid = '" + user_id + "' and userpswd = '" + password + "'";

query = "SELECT 1 FROM USER_TABLE WHERE userid = '" + user_id + "' and userpswd = '" + password + "'";

query = "SELECT userpswd FROM USER_TABLE WHERE userid = '" + user_id + "' and userpswd = '" + password + "'";

---

snPassword.getInstance().encrypt(password) was specifically written incorporating base64 algorithm for encrypting a string.  However, the decryption is cumbersome and therefore, comparison of passwords are not made by decrypting the stored value of the password; rather is done by encrypting the input password using the same method and comparing the encrypted password with that stored in the database table.

---

For better standards, no decrypt method was provided in the snPassword class so that coders would it only as an ecrypting tool and not for decrypting.

However, later on (around Dec. 2010), requirement came in for having a decrypting algorithm.  I had in 1998 - 2000 an algorithm developed for an Imprest Module that was written in PowerBuilder and worked well.  Therefore, the same algorithm was copied into a new Java class called snEncryption.

This class provided encrypt() and decrypt() algorithms.  However, what I like most about this class is an additional feature that you might not see many-where!!.

The feature was developed in response to a requirement of communicating securely between applications on HTTP.  In the normal course, if we need to communicate between applications, say call an http page from one application to another, we could just pass it on creating a URL with the & delimiter joining the various parameters being passed.  How then, is it possible that the receiving application understands that the requesting URL is not arriving from any hacker trying to get some sensitive details?  In such a case, the following technique can be used.

This interesting feature can encrypt a URL with its parameters and send to another web-application.  The receiving web-application should have included a small filter inside its WEB-INF/web.xml that would ensure that the original parameters are received by the appropriate servlet. The feature is enriched by the fact that the life of the URL is not more than (default) 180 seconds.

Needless to say, that this methodology, can also be adopted as an internal security feature wherein, referring URLs within the same application could also be done with this alone.

Let me show you the implementation.
Dispatching Web Application:
File:-myURLDispatcher.jsp
<%@page import="config.snEncryption" %>
// ...
// ... Some part of the Servlet Code goes here 
// ...
<%
String param = "myparam1=First Param&myparam2=Second Param&noparam=No Param&userid=Kavya";
snEncryption snE = new snEncryption(param.length());
String encryptedParam = snE.encryptURLtoURLParam("NewServlet?", param, null, true);  
// Null is default value for UTF-8 encoding with default TTL (Time To Live) of 180 seconds
//            -or-
//  String encryptedParam = snE.encryptURLtoURLParam("NewServlet?", param, null, true, 120);  
// Last parameter indicates TTL of 120 seconds instead of default 180 seconds.
%>
  <form name=form1 method=get action=<%=encryptedParam%>  >
      <input type=submit value=Submit />
  </form>

Note:-  The NewServlet.java would use the request.getParameter("userid") to get the value "Kavya", use request.getParameter("myparam1")  to get the value "First Param" etc.  In other words, there is no need for decrypting the URL in the receiving servlet because, the decryption algorithm is taken care of by the Filter.

File:WEB-INF/web.xml 
<filter>
<filter-name>Induce Params Screen Approver</filter-name>
<filter-class>org.sun.raibmanus.filters.snInduceParamsScreenApprover</filter-class>
    <init-param>
      <param-name>exclude_url-2</param-name>
      <param-value>/folder/index_[3-5].jsp</param-value>
    </init-param> 
    <init-param>
       <param-name>exclude_url-4</param-name>
       <param-value>([\w||\-||/])+\.([gG][iI][fF]|[pP][dD][fF]|[pP][nN][gG]|[jJ][pP][gG]|[jJ][pP][eE][gG]|[bB][mM][pP]|[jJ][sS]|[cC][sS][sS]|[sS][wW][fF]|[iI][cC][oO])</param-value>
    </init-param>        
    <init-param>
      <param-name>include_url-2</param-name>
      <param-value>/folder/index_4.jsp</param-value>
    </init-param> 
</filter>

<filter-mapping>
<filter-name>Induce Params Screen Approver</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

Apart from encryption, this filter also allows including in all the Http requests, persistent  request parameters that are stored in the snConfigFile.  For example, suppose the following should be a forced parameter in all requests of my web-application which I could use the getParameters() method to access, we may define them in the snConfigFile as below.

File:=c:/snConfig/snConfigMyApplication.ini

Persistent Request Parameter-name=Suresh Nambiar
Persistent Request Parameter-userid=Jack
Persistent Request Parameter-pwd=password233453
Persistent Request Parameter-Pwd=sureshpassword

What this means is that along with each and every request coming into the web-application server, the above four parameters would also get appended.  The four parameters in this case are 'name', 'userid', 'pwd' and 'Pwd'.

If org.sun.raibmanus.filters.snInduceParamsScreenApprover is used for the filter, the received parameters if any, are also included along with the Persistent Request Parameters.  Whereas in the case of org.sun.raibmanus.filters.snIncludeParamsFilter, no existing values other than those sent as encrypted using snEncryption..encryptURLtoURLParam() & the Persistent parameters are included. (org.sun.raibmanus.filters.snIncludeParamsFilter is deprecated.)

Yet another improved version (version 11.1 dtd. 24th Dec. 2010) has additional features viz. removing from response content, remove certain content types from the response text like patterns that contain 'xml', 'text', 'text/html' etc.  org.sun.raibmanus.filters.snInjectedParamsResponsePatternModifier can be used as the filter in-lieu of the above mentioned snIncludeParamsFilter.  

If the snInjectedParamsResponsePatternModifier is used, the filter shall be as follows:

File:-WEB-INF/web.xml 

<filter> 

<filter-name>Injected Params Response Pattern Modifier</filter-name>
<filter-class>org.sun.raibmanus.filters.snInjectedParamsResponsePatternModifier</filter-class>
<init-param>
<param-name>ModifyRequestQueryString</param-name>
<param-value>1</param-value>
</init-param>
<init-param>
<param-name>RemoveFromResponseContent</param-name>
<param-value>1</param-value>
</init-param>
<init-param>
<param-name>ContentTypeforRemoveResponse-1</param-name>
<param-value>text/html</param-value>
</init-param>
<init-param>
<param-name>ContentTypeforRemoveResponse-2</param-name>
<param-value>text</param-value>
</init-param>
<init-param>
<param-name>ContentTypeforRemoveResponse-3</param-name>
<param-value>xml</param-value>
</init-param>
</filter>

<filter-mapping>
<filter-name>Injected Params Response Pattern Modifier</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

And in the snConfigFile, the entry should be as follows:
File:- c:/snConfig/snConfigMyApplication.ini
Persistent Request Parameter-name=Suresh Nambiar
Persistent Request Parameter-userid=Jack
Persistent Request Parameter-pwd=password233453
Persistent Request Parameter-Pwd=sureshpassword

#Response Content Search Pattern=http//
#Response Content Search Pattern=viewer/viewframeset/
Response Content Search Pattern=.*

#Response Content Replace Pattern==[^&]*&?
#Response Content Replace Pattern==[a-e]*
Response Content Replace Pattern==[^&]*&?\s

Note that there can only one Content Search and Replace Pattern. Only the last valid entry (ie. without the starting # mark) would prevail.

With snLibrary version 11.1,  org.sun.raibmanus.filters.snIncludeParamsFilter,  org.sun.raibmanus.filters.snInduceParamsScreenApprover  etc. have been deprecated.

Note:- The 'Response Content Search Pattern' + 'Response Content Replace Pattern' work together and is operational if and only if RemoveFromResponseContent is set to 1 in the initparam. In which case, the response data is filtered and replaced by the pattern specified.

There is an incredible story behind providing this feature.  We had a reporting software called 'Actuate'.  It works on a Tomcat web-server providing compiled reports.  The required reports are compiled in the night and posted for ready availability giving the impression of a fast retrieval thereby improving user-experience.  For making the reports available to the internet, the Actuate server was put on NAT (Network Access Translation).  Once, it so happened that some('one') from the internet world (China or Korea...) hacked-in and deposited a bot into it.  It was then decided that some security needs to be improved on the Actuate server as it was exposed to Public IP addressing. Therefore, I suggested to put the Actuate server behind the reverse proxy and started exploring the source codes.  One day, I was examining the source of the response received from the Actuate server when to my surprise, found that the User-ID  as well as the password was being thrown down to each page as a hidden input field so that every further request could carry it along and get authenticated at the Actuate Server.  (Let me tell you, that day I was shocked to see the poor Code Quality that was unexpected but exhibited promptly, by well known and established Multi-National Companies like the one that was examined).  It was further surprising to find that it was the Admin user id with all authorities & rights along with its password for the Actuate Server that was so foolishly exposed and communicated as response data to the client!!.  I immediately alerted the team responsible for hosting the Actuate Server.  This has however, now resulted in a better configuration with the response being camouflaged (at least) to keep away not-so savvy ones from peeping into the Actuate Server - though definitely neither the amateur nor the experienced hackers.

I wanted to do something for this cause and therefore, included this specific Filter mentioned above that would pickup the user id and password from the Actuate Server itself (as Persistent Request Parameters) while not sending the values back to the client (by filtering them out through the 'Response Content Search Pattern' + 'Response Content Replace Pattern' combination).

---

The last part would ensure that you do get cleaner response data with some sort of filter applied.  Such are usually useful in AJAX responses and ensuring accuracy in or validating  the response at the server side itself.

---

snLibrary - Inducing External Values to avoid Hard Coding

snLibrary - Inducing External Values

One of the guiding principles that is part of the philosophy of 'snLibrary' is to maintain zero-hard-coding in application development.

I have many times noticed while web-application development, specific requirements of 'hard-coding' does come in which are hard to reject.  One such example would be the requirement to hard-code URLs or IP addresses or domain names of other servers or services that you need to put into the application code for URL re-direction or for providing hyperlinks. When such situation arises, what can we do?   The easiest method is either to 'hard-code'!! or save them up in a Java 'class' or 'interface' and implement them whenever needed.  But it would be at the cost that whenever there arises a request for change, you would need to modify the code and re-deploy in production.  Another method would be to set them up as context parameters in the web.xml and use it within the program.  Here again, whenever a change is required, the web-administrator would  need to actually extract the web.xml file out of the web-archive (.war) file and then modify and re-deploy the same.  This requires man-hours and technical expertise and careful handling. While in the tomcat environment this might be pretty easy, in JBoss or other environments and wherever exploding web archive file option is disabled, it would entail recompiling of the .war file.

Shall we avoid such extensive operations by any means?  Can snLibrary provide some solution?  This feature is available since version 22.2 dtd. 14-Feb-2015.

The following methodology can be used as a solution to the above problem.

Let us assume that we need to specify certain variables (or say constants - in this case) that needs to form prefix to few hyperlinks that your application would refer to.  Let us define the variable names as http.1, http.2, http.3, http.4, http.5, http.6.  (Needless to mention that the name of the variable/constant could be anything of your choice - say it could be alpha, beta, gama or say main.background-color, main.foreground-color etc.).  Another variable called 'defaultLanguage' is also mentioned for better understanding.

File:-c:/snConfig/snConfigMyApplication.ini
External Variable-http.1=http://10.34.133.216:8084/TimeAttendanceNew
External Variable-http.2=http://10.75.132.31
External Variable-http.3=http://10.34.132.119
External Variable-http.4=http://10.75.132.30:8080
External Variable-http.5=http://10.18.64.84
External Variable-http.6=http://10.75.132.15
External Variable-defaultLanguage=Hindi

 In the Java code, we could refer these variables/constants as follows:
File:- someJavaCode.java
import config.snConfigVars; 
// ...
// .. Some part of code goes here....
// ..
snConfigVars V = snConfigVars.getInstance();
String urlMypath = V.getExternalVariable("http.4", "");
// Thus 'urlMypath' would contain the value 'http://10.75.132.30:8080'

String lang = V.getExternalVariable("defaultLanguage", "English"); 
// Here, 'lang' would contain the value 'Hindi'.  In case, the variable was not provided in snConfigFile, it would have taken it as 'English' for default.

String bkg_color = V.getExternalVariable("main.background-color", "#FFFF");
// Thus the value of bkg_color would be #FFFF since no value has been specified in the snConfigFile.

---

Having found this very useful, I would rather also use it to create some other features - say create a web-page which is language sensitive.  For this, I could define as follows:-

File:-c:/snConfig/snConfigMyApplication.ini
External Variable-defaultLanguage=hi
#External Variable-mainpage.name.en=Name
#External Variable-mainpage.age.en=Age
#External Variable-mainpage.dob.en=Date of Birth
#External Variable-mainpage.addr.en=Address
External Variable-mainpage.name.hi=नाम
External Variable-mainpage.age.hi=आयु
External Variable-mainpage.dob.hi=जन्म तिथि
External Variable-mainpage.addr.hi=पता

And in the mainpage,jsp, the form could be something as follows.

File:-mainpage.jsp
<%@page contentType="text/html" pageEncoding="UTF-8"%>
<%@page import="config.snConfigVars" %>
<%
snConfigVars V = snConfigVars.getInstance();
String lang = V.getExternalVariable("defaultLanguage", "en"); 
%>
<html>
<body>
<form action="nextpage.jsp">
    <table border='0'>

    <tr><td> <%= V.getExternalVariable("mainpage.name." + lang, "Name")%> 
    </td><td> <input name="username" type="text" />
    </td></tr>

    <tr><td> <%= V.getExternalVariable("mainpage.age." + lang, "Age")%> 
    </td><td> <input name="age" type="text" />
    </td></tr>

    <tr><td> <%= V.getExternalVariable("mainpage.dob." + lang, "Date of Birth")%> 
    </td><td> <input name="d_o_b" type="text" />
    </td></tr>

    <tr><td> <%= V.getExternalVariable("mainpage.addr." + lang, "Address")%> 
    </td><td> <input name="addr" type="text" />
    </td></tr>

    </table>
     <input type="submit" name="Submit" value="Submit" />
</form>
</body>
</html>

Better still, if it was a case of Language properties, it could also be in an external properties file.  In that case, it could be mentioned in the snConfigFile as follows and we can move all the definitions of external variables into the external file.

File:-c:/snConfig/snConfigMyApplication.ini

External Variable Properties File=c:/snConfig/snExternalVariablesProperties.properties

File:-c:/snConfig/snExternalVariablesProperties.properties

defaultLanguage=hi

mainpage.name.en=Name
mainpage.name.hi=नाम
mainpage.name.sn=नाम: 

mainpage.age.en=Age
mainpage.age.hi=उम्र
mainpage.age.sn=आयु:

mainpage.dob.en=Date of Birth
mainpage.dob.hi=जनम दिन
mainpage.dob.sn=जन्म दिनाङ्कः

mainpage.addr.en=Address
mainpage.addr.hi=पता
mainpage.addr.sn=सङ्केत:

---

snLibrary - Securing your Web-Application

snLibrary - Securing your Web-Application

While connection pooling is one aspect of a healthy application, as you would have already seen in my earlier blog,  can we have a simple method for some sort of Application Authentication security?  Certainly yes.  If so, is it possible to achieve this with very little code?.

Can snLibrary help? 

As we all know, application authenticity is established by way of challenging a user to authenticate himself/herself at the time of login.  Once he/she logs in to the application, the web-container maintains a unique session for the link between the user and the application.  As a keen observer, you would have often seen for yourself sometime during your childhood when you copied and pasted a URL (that was verily displayed on the address bar of your lovely browser) onto another browser or otherwise copied it and sent to your friend by email.  And Lo!! have you not many times become very happy seeing that the page opened before you just as it would have, had you actually gone through the tiring process of Login to the application?  If such an action did work perfectly well, then I would rather say that this certainly was (at least if it wasn't published intentionally), a poor programmer who did bad coding. 

As an intelligent coder, you would certainly want to get out of such misadventure, if at all any time you experienced such a 'fault' in your own code (which are those rare events when you wanted to hide yourself from being laughed at!!).   Some of you might have instructed your coders to put a line of code for verification in each of the web-page, wherein each JSP page would be verified for a session attribute, say user-id or so, before actually letting the code display the web-page. Yes indeed - let me also acknowledge that it is the right way!!. However, as it is often said, there is a lot between theory and practice.  We have already seen that in theory we always would like our code to open and close all database connections in each page - but does that happen in practice?  The answer is:- many times not.  (Needless to say, many times while your coders would have done the verification in each web-page, they would have forgotten to put in the servlets or at least otherwise in those AJAX (Asynchronous Java And XML) called codes).

To overcome this, a small feature was provided in the snLibrary (since Ver-10.0 dtd. 20th Aug. 2010) which I named it the Indirect entry Redirection option.  It means that if per chance, someone tries to access any page through 'indirect' means!!,  then it would be redirected to the so called 'login' page. Such a feature can be implemented as follows.

Let us assume that there is one Login page and a Validation page/servlet.  The Login page would be usually the welcome page of the Web-application (defined under the welcome tags of web.xml).  Often, it is the page called 'index.jsp' or 'index.html' or any other that is defined in the web.xml under the 'welcome' tag.  The Login page could be the one that would accept the user-id and password.  Having accepted the credentials of the user, it would pass on to the Validation page. The validation page would validate and then if the credentials were acceptable, would proceed to other pages; otherwise take it back to the Login page (often with an error message). 

User ------>  Login.jsp ---> Validate.jsp -- if OK ----> ApplicationMainPage.jsp

User ------>  Login.jsp ---> Validate.jsp -- if NOT OK ----> Login.jsp

                                 

However, there could be other pages which as a programmer you would wish to intentionally leave open without requiring to be authenticated as well.  Such cases can be put in the 'exclusion' pattern list.

Thus we have three parameters namely:-  the login page which I shall call the 'default_url',  the validation page which I shall call the 'validator_url' and one or more pages that do not require any authentication (viz. the .bmp, .js, .css, .png, .gif, .jpg etc.) which I shall call the 'exclude_url-nnn'.  These three patterns are defined as parameters in the web.xml under 'Indirect Entry Redirection Filter'. Include the following in the web.xml file. 

File:- WEB-INF/web.xml

  <filter>
    <filter-name>Indirect Entry Redirection Filter</filter-name>
    <filter-class>sessions.snIndirectEntryRedirectorFilter</filter-class>
    <init-param>
      <param-name>default_url</param-name>
      <param-value>login.jsp</param-value>
    </init-param>
    <init-param>
      <param-name>validator_url</param-name>
      <param-value>login_validate.jsp</param-value>
    </init-param>  
    <init-param>
      <param-name>exclude_url-1</param-name>
      <param-value>index_2.jsp</param-value>
    </init-param>
 <!-- Any webpage starting with /myfolder/index_, following with 3, 4 or 5 followed by a character and then jsp -->
    <init-param>
      <param-name>exclude_url-2</param-name>
      <param-value>myfolder/index_[3-5].jsp</param-value>
    </init-param>        
<!--  Webpage starting with /myfolder/index_4.jsp, even though excluded by /myfolder/index_[3-5].jsp would get included -->
    <init-param>
      <param-name>include_url-2</param-name>
      <param-value>myfolder/index_4.jsp</param-value>
    </init-param> 
<!-- Any word-character or dash (-) or slash (/) any number of times with extension gif,pdf,jpg etc.  -->
    <init-param>
      <param-name>exclude_url-3</param-name>
      <param-value>([\w||\-||/])+\.(gif|pdf|png|jpg|jpeg|bmp|js|css|swf|ico)</param-value>
    </init-param>       
<!-- Any word-character or dash (-) or slash (/) any number of times with extension GIF,PDF,PNG,JPG etc.  -->
    <init-param>
      <param-name>exclude_url-4</param-name>
      <param-value>([\w||\-||/])+\.([gG][iI][fF]|[pP][dD][fF]|[pP][nN][gG]|[jJ][pP][gG]|[jJ][pP][eE][gG]|[bB][mM][pP]|[jJ][sS]|[cC][sS][sS]|[sS][wW][fF]|[iI][cC][oO])</param-value>
    </init-param>       
  </filter>
  
  <filter-mapping>
    <filter-name>Indirect Entry Redirection Filter</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>  

 For this to work, there is just one more code to be put in the VALIDATED='TRUE' condition of the validator_url which is given below:-

File:-login_validate.jsp

import sessions.snAuthentications;

// ....

// ....

// ... Some part of the code goes here inside the Validate.jsp 

// ...

   if (valid_user_id) { 

        snAuthentications.properlyLoggedIn(request, response);

        // ...

        // ... Some other part of the code 

       // ....

   }

// ....
// ... Some other part of the code goes here 
// ...

---