Monday, December 17, 2012

How To Encrypt Clear Text Passwords With WebLogic Server

WebLogic Server encrypts all the plain text passwords stored in its domain configuration XML file(s). This is to prevent access to sensitive information. When passwords are entered using administration console or scripting tools, it will automatically get encrypted before they are stored in the configuration XML files(s).

Prior to WebLogic Server 9.0

If those passwords need to be reset either the configuration tools (Console or scripting tools) can be used which will automatically re-encrypt the passwords or by directly changing the configuration files using a text editor. When files are directly modified using a text editor the passwords will get encrypted during the subsequent restart.

Starting from WebLogic Server 9.0

Using clear text passwords in the configuration files are supported only for Development domain and it will not re-encrypt the passwords. If the domain is a Production domain then you cannot set the passwords in clear text. You have to either use a dedicated command-line utility or WLST to encrypt the clear text passwords. If the server encounters a clear text password when parsing the configuration file(s) while starting in Production Mode, then you will get an error similar to the following:

<Oct 20, 2008 9:05:35 PM EDT> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason: [Management:141266]Parsing Failure in config.xml: java.lang.IllegalArgumentException: In production mode, it's not allowed to set a clear text value to the property: PasswordEncrypted of ServerStartMBean>


Depending on the configuration the MBean name value of the error message may change. In this case the ServerStartMBean has clear text value for a password property. Either the dedicated Java utility to encrypt clear text values can be used or WLST cant be used to re-encrypt. To run the encrypt utility follow the instructions below:


  1. Change directory to your domain's bin folder (For Eg. cd c:\bea\user_projects\domains\mydomain\bin)
  2. Execute the setDomainEnv script (For Eg. setDomainEnv.cmd)
  3. Execute java weblogic.security.Encrypt which will prompt for the password and will print the encrypted value in stdout.
The following are some sample output from running the utility

C:\bea\user_projects\domains\mydomain>java weblogic.security.Encrypt
Password:
{3DES}9HWsf87pJTw=

You should execute this utility from the domain folder as it requires the domain's password salt file (SerializedSystemIni.dat) for encrypting the clear text string. You can also pass the clear text string as an argument

C:\bea\user_projects\domains\mydomain>java weblogic.security.Encrypt testpwd
{3DES}9HWsf87pJTw=


You can also use WLST to encrypt clear text strings as below:

C:\bea\user_projects\domains\mydomain>java weblogic.WLST

Initializing WebLogic Scripting Tool (WLST) ...

Welcome to WebLogic Server Administration Scripting Shell

Type help() for help on available commands

wls:/offline> es = encrypt('testpwd')
wls:/offline> print es
{3DES}9HWsf87pJTw=
wls:/offline>

When running WLST from a location different than the domain folder you can pass in an argument to specify the domain directory. Once you have the encrypted value, the configuration files can be modified to include this encrypte value instead of clear text passwords. These features will make your domain to operate when resetting the encrypted passwords on a Production domain's configuration XML files. These methods not only can be used to encrypt configuration XML (config.xml) but also the JDBC or JMS descriptor XML files.

Difference between Development Mode and Production Mode in Weblogic server

During domain creation you can specify the start up mode for your domain either as development mode of production mode. 

Most of you who work with WebLogic Server for the past few releases should know that there are few differences between a development domain and production domain.
(http://e-docs.bea.com/common/docs103/confgwiz/newdom.html#wp1097267

Development Mode
The default JDK for development domain is Sun Hotspot
You can use the demo certificates for SSL
Auto deployment is enabled
Server instances rotate their log files on startup
Admin Server uses an automatically created boot.properties during startup
The default maximum capacity for JDBC Datasource is 15
The debugFlag which is used to start the WebLogic Workshop Debugger is enabled

Production Mode
The default JDK for production domain is JRockit
If you use the demo certificates for SSL a warning is displayed
Auto deployment is disabled
Server instances rotate their log files when it reaches 5MB
Admin Server prompts for username and password during startup
The default maximum capacity for JDBC Datasource is 25
The debugFlag which is used to start the WebLogic Workshop Debugger is disabled

In addition to the above WebLogic Server 10gR3 adds a few more default configurations depending on whether the domain is started in development or production mode.



Common Issues and solutions to errors during Apache HTTP Server startup


1. Apache HTTP Server startup issue.

2. Unable to accept incoming connections in a server machine.
 1. Apache HTTP Server startup issue.
 This is a very standard issue which is noticed while starting the ApacheHTTPServer?
service on windows. The possible reasons could be the following services. These services, if they are running, will occupy the port 80 in your machine and as a result you will not be able to start the ApacheHTTPServer  service.
a. IIS – Just stop this service before trying to start Apache on port 80 in your machine.
b. Skype – This seems to occupy the port 80 in your machine by default. If you have to use Skype in your machine, please make sure you have reconfigured your Skype to not occupy port 80.
c. OracleHTTPServer? – This service also should be stopped before starting the Apache on your machine
2. Unable to accept incoming connections in a server machine
a. Windows XP has some builtin firewall settings where all the incoming connections can be blocked. Disable this firewall setting or add the relevant port numbers as exceptions in the firewall setting.
If this is not disabled, you cannot access the application installed on the server machine.
 3. On Unix is related issue in running Apache as user root:
1. When we start Apache as “root”, Apache goes by the “User” directive in the Apache config file httpd.conf to spawn the httpd processes.
To check as what user httpd is running simply type ps -elf | grep httpd and you will see the owner of httpd (the apache process). Or you can look at owner of the httpd log files inside Apache/logs directory.
2. By default, this user will be set to username “nobody”, which will be present in most machines, but will not have enough permissions most of the time.
3. Typically “nobody” user should have access permissions to the Apache directories, if not you will get this “Forbidden error” on the page.

unpack Command Throws a NullPointerException | Weblogic


In some servers, a NullPointerException is thrown while running the unpack command.
The pack command used to populate the jar file did not complete successfully while executing .
Let's have a example, if the pack command were run while the admin and/or managed servers of the domain were still running, the pack command would not be able to complete, and the jar file it builds would not be complete either. 

Then if the same incomplete jar file is used for the unpack command, it will lack some of what it needs, and the error will occur.
Make sure that the pack command has completed before taking the jar file and using it for unpack.

In the example, make sure that the admin and managed servers of a domain have been shut down before running the pack command so that it can complete. After it has completed, the jar file will be usable and then it can be used to unpack.

crashing jrockit while starting WebLogic server instance


Just today I spent some time working on a JVM (jrockit) while starting WebLogic with the default startWebLogic.sh script. The enviroment is rather straight forward:
  • Jrockit (jrockit-jdk1.6.0_22-R28.1.1-4.0.1)
  • WebLogic 10.3.4
  • SOA Suite PS3 (11.1.1.4)
Now whenever I started WebLogic I ran into a rather lengthy core dump, which you can view: here (just rename to .txt). The interesting part of this core dump is this:


Loaded modules:
(* denotes the module where the exception occured)
0000000000400000-0000000000412703 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/bin/java
0000003353200000-0000003353201f93 /lib64/libdl.so.2
0000003353600000-0000003353615a6f /lib64/libpthread.so.0
0000003352a00000-0000003352b4d067 */lib64/libc.so.6
0000003351a00000-0000003351a1bb7f /lib64/ld-linux-x86-64.so.2
00002ba4453b2000-00002ba4456b734b /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/jrockit/libjvm.so
00002ba445991000-00002ba4459af9eb /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libosal.so
00002ba445bb2000-00002ba445bbe4fb /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libutil.so
0000003352e00000-0000003352e8122f /lib64/libm.so.6
0000003354600000-0000003354606f5b /lib64/librt.so.1
00002aaaaabb4000-00002aaaaabe9c17 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libjdwp.so
00002aaaaacf3000-00002aaaaacf5307 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libnpt.so
00002aaaae3d0000-00002aaaae3dbd03 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libjfr.so
00002aaaf1a92000-00002aaaf1a9e337 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libverify.so
00002aaaf1ba1000-00002aaaf1bc8c73 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libjava.so
0000003355200000-00000033552140c7 /lib64/libnsl.so.1
00002aaaf1d10000-00002aaaf1d165bf /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/native_threads/libhpi.so
00002aaaf22dc000-00002aaaf22e98cb /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libzip.so
00002aaaf2530000-00002aaaf2532e73 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libdt_socket.so
00002aaaf2643000-00002aaaf264c64b /lib64/libnss_files.so.2
00002aaaf2a4e000-00002aaaf2a6098f /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libnet.so
00002aaaf3b26000-00002aaaf3b2bf43 /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/liborii.so
00002aaaf3ded000-00002aaaf3df38ab /u01/app/oracle/product/1.6.0/jrockit-jdk1.6.0_22-R28.1.1-4.0.1/jre/lib/amd64/libnio.so
00002aaaf8e80000-00002aaaf8e818dd /u01/app/oracle/product/11.1.1.4/wlserver_10.3/server/native/linux/x86_64/libweblogicunix1.so

Now this gave me a hook to start querying google and metalink, ending up with only one advice: downgrade jrockit. Obviously I tried this, but this and it gave me the root cause:

<Apr 12, 2011 2:47:54 PM CEST> <Critical> <WebLogicServer> <BEA-000252> <Cannot switch to group "dba". java.lang.IllegalArgumentException: Cannot setegid("dba"), system error: ''
java.lang.IllegalArgumentException: Cannot setegid("dba"), system error: ''
at weblogic.platform.Unix.setEGroup0(Native Method)
at weblogic.platform.Unix.setEffectiveGroup(Unix.java:73)
at weblogic.t3.srvr.SetUIDRendezvous.setEGroup(SetUIDRendezvous.java:156)
at weblogic.t3.srvr.SetUIDRendezvous.makeUnPrivileged(SetUIDRendezvous.java:174)
at weblogic.t3.srvr.SetUIDRendezvous.initialize(SetUIDRendezvous.java:88)
at weblogic.t3.srvr.BootService.start(BootService.java:48)
at weblogic.t3.srvr.ServerServicesManager.startService(ServerServicesManager.java:461)
at weblogic.t3.srvr.ServerServicesManager.startInStandbyState(ServerServicesManager.java:166)
at weblogic.t3.srvr.T3Srvr.initializeStandby(T3Srvr.java:879)
at weblogic.t3.srvr.T3Srvr.startup(T3Srvr.java:566)
at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:467)
at weblogic.Server.main(Server.java:71)
>

Now this did help me. Suddenly I realized that I had been playing with machine configuration (uid and gid setting to be more specific)

<machine xsi:type="unix-machineType">
<name>testbox</name>
<node-manager>
<name>testbox</name>
<listen-address>localhost</listen-address>
</node-manager>
<post-bind-uid-enabled>true</post-bind-uid-enabled>
<post-bind-uid>testuser</post-bind-uid>
<post-bind-gid-enabled>true</post-bind-gid-enabled>
<post-bind-gid>dba</post-bind-gid>
</machine>

Now testuser doesn’t have dba as primary group:

$ id testuser
uid=10641(testuser) gid=100(users) groups=4006(dba),100(users)

So because of this jrockit crashes… (wrong group for the machine). The error is kind of unfriendly, downgrading seems to be a way of improving error messages.
Hope this helps.

Configuring Fusion Middleware JDBC Data Sources Correctly


The out of the box settings for a data source in a random Fusion Middleware product (SOA, WebCenter, OIM, etc. they are all alike) JDBC properties contains guesses about your environment and usage.
Same goes for the settings required by RCU when installing a repository.
For a customer I recently wrote a document explaining which settings to set on the database and in WebLogic when configuring data sources for a Fusion Middleware product for production usage while connected to a RAC database.
The document assumes you are running a 11.2 RAC and WebLogic 10.3.4 or newer. Here’s the document:
Hope this helps.
BTW: if you already downloaded the document, please download it again. Seems I made an error in the distributed lock area.

Different Memory Argument values for SOA Servers and BAM Servers In Weblogic 11g.



This post is about providing specific memory arguments values to SOA server and BAM servers in Weblogic 11g.

There are two scenarios:

  1. Admin Server is Started using the startWeblogic.cmd or startWeblogic.sh file and all other servers are started using startManagedWeblogic.cmd or startManagedWeblogic.sh 

  2. Admin Server is Started using the startWeblogic.cmd or startWeblogic.sh file and all other servers are started using the Admin Server Console.

Case1 :  

Admin Server is Started using the startWeblogic.cmd or startWeblogic.sh file and all other servers are started using startManagedWeblogic.cmd or startManagedWeblogic.sh 

First of all we have to analyse the startWeblogic.sh file that is used to start the Admin Server and the startManagedWeblogic.sh file that is used to start all other managed servers such as SOA server and BAM server.
Both the above mentioned file uses setDomainEnv.sh file to set the Domain related environment.
And setDomainEnv.sh file uses setSOADomainEnv.sh file for setting SOA specific environments.
So the memory related arguments like Xmx and Xms are picked up from the file :setSoaDomainEnv.sh file present in the $DomainHome/bin directory.
So, in order to provide different Xmx and Xms values for Admin Server , SOA server and BAM server we have to modify the setSOADomainEnv.sh file as shown below :
Imp : Please make sure to take backup of all the original files before modifying them .
In setSOADomainEnv.sh file comment the below lines :
# DEFAULT_MEM_ARGS="-Xms2048m -Xmx2048m"
# PORT_MEM_ARGS="-Xms2048m -Xmx2048m"
Insert the below lines replace the above commented code :
# line to specify different Xmx and Xms values to soa_server and bam_server.

if [ "${SERVER_NAME}" = "soa_server1" ] || [ "${SERVER_NAME}" = "soa_server2" ]; then
DEFAULT_MEM_ARGS="-Xms2048m -Xmx2048m"
PORT_MEM_ARGS="-Xms2048m -Xmx2048m"
elif [ "${SERVER_NAME}" = "bam_server1" ] || [ "${SERVER_NAME}" = "bam_server2" ]; then
DEFAULT_MEM_ARGS="-Xms2048m -Xmx2048m"
PORT_MEM_ARGS="-Xms2048m -Xmx2048m"
elif [ "${SERVER_NAME}" = "" ] || [ "${SERVER_NAME}" = "AdminServer" ]; then
DEFAULT_MEM_ARGS="-Xms2048m -Xmx2048m"
PORT_MEM_ARGS="-Xms2048m -Xmx2048m"
else
DEFAULT_MEM_ARGS="-Xms2048m -Xmx2048m"
PORT_MEM_ARGS="-Xms2048m -Xmx2048m"
fi

# end of code

Case 2 :  

Admin Server is Started using the startWeblogic.cmd or startWeblogic.sh file and all other servers are started using the Admin Server Console.

In this case, the Memory Arguments can be provided to from the Admin Server Console using the below steps :

  •  Login into the Weblogic Administration Console using URL :http://admin_server_hostname:port/console
  • Expand Environment Tree available on the left hand panel of the Admin Console.
  • Click on Servers .
  • For each Server say ( soa_server1 , and bam_server ) click on the server name .
  • Then click on the Configuration tab.
  • Then go to the Server Start tab available on the top of the page.
  • Add the memory values in the Arguments field as shown below. There is no need to use newline. You can provide as many memory options as you want in this area and all the options will be separated by one white space character only.

-Xmx2048m -Xms2048m
  • Click Save