Error while loading shared libraries: libXm.so.3: cannot open shared object file

Symptoms:
During the installation of Oracle Fusion Middleware Portal Forms Reports or the compilation of forms the utilities frmcmp or frmbld cant find the libs to execute.

Cause: 
The executable is looking for a shared library libXm.so.3 in 64 bits because OAS 11g software distribution being installed is 64 bits and the current shared library on /usr/lib/libXm.so.3 is from a 32 bits RPM package.
This lib belongs to openmotif lib packages, so you have to validate if this is the case - libs not installed in your system:

Eg via rpm:
rpm -q --queryformat "%{NAME}.%{ARCH}\n" openmotif

rpm -q --queryformat "%{NAME},%{ARCH}\n" openmotif22


Eg via yum:

yum info openmotif

yum info openmotif22


Solution:
Make sure the following 64 bits openmotif packages are installed:
Example for RedHat 4.

openmotif-2.2.3-10.5.el4.x86_64.rpm
openmotif22-2.2.3-18.x86_64.rpm

You can also check on the Oracle Fusion installation guide, section pre-requirements will give you a list of all libraries that you need installed on your OS.
https://www.oracle.com/technetwork/middleware/ias/downloads/fusion-requirements-100147.html

From above link, the libraries needed to be installed on the OS are:


Oracle Linux 7 (UL0+)
Red Hat Linux 7 (UL0+)

yum install -y binutils-2.23.52.0.1
yum install -y compat-libcap1-1.10
yum install -y compat-libstdc++-33-3.2.3 for x86_64
yum install -y compat-libstdc++-33-3.2.3 for i686
yum install -y gcc-4.8.2
yum install -y gcc-c++-4.8.2
yum install -y glibc-2.17 for x86_64
yum install -y glibc-2.17 for i686
yum install -y glibc-devel-2.17 for x86_64   
yum install -y glibc-devel-2.17 for i686
yum install -y libaio-0.3.109 for x86_64
yum install -y libaio-devel-0.3.109 for x86_64
yum install -y libgcc-4.8.2 for x86_64
yum install -y libgcc-4.8.2 for i686
yum install -y libstdc++-4.8.2 for x86_64
yum install -y libstdc++-4.8.2 for i686
yum install -y libstdc++-devel-4.8.2 for x86_64
yum install -y ksh make-3.82 ocfs2-tools-1.8.0
yum install -y libXext-1.3.2 for i686
yum install -y libXext-1.3.2 for x86_64
yum install -y libXtst-1.2.2 for x86_64
yum install -y libXtst-1.2.2 for i686
yum install -y sysstat-10.1.5
yum install -y motif-2.3.4-7 for x86_64
yum install -y motif-devel-2.3.4-7 for x86_64
yum install -y motif-2.3.4-7 for i686
yum install -y motif-devel-2.3.4-7 for i686
yum install -y redhat-lsb-4.1-27.0.1.el7 for x86_64
yum install -y redhat-lsb-core-4.1-27.0.1.el7 for x86_64

For RH/OL 7, it might be missing a symbolic link for version 3. See below:
$ ls -la /usr/lib64/libXm.so*
/usr/lib64/libXm.so -> libXm.so.4.0.4
/usr/lib64/libXm.so.4 -> libXm.so.4.0.4

/usr/lib64/libXm.so.4.0.4

cd /usr/lib64/

ln -s /usr/lib64/libXm.so.4.0.4 libXm.so.3


$ ls -la /usr/lib64/libXm.so*
/usr/lib64/libXm.so -> libXm.so.4.0.4
/usr/lib64/libXm.so.3 -> /usr/lib64/libXm.so.4.0.4
/usr/lib64/libXm.so.4 -> libXm.so.4.0.4

/usr/lib64/libXm.so.4.0.4


Append /usr/lib64 to all LD_LIBRARY_PATH referenced in $ORACLE_HOME/bin/config.sh, like so:

if [ x${LD_LIBRARY_PATH} != x ]; then                 LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH:/usr/lib64

 export LD_LIBRARY_PATH

else

 LD_LIBRARY_PATH=$ORACLE_HOME/lib:/usr/lib64 

 export LD_LIBRARY_PATH

Run the config.sh from $ORACLE_HOME/bin.

Oracle Linux 6 (UL1+)
Red Hat Linux 6 (UL1+)

yum install -y binutils-2.20.51.0.2-5.28.el6
yum install -y compat-libcap1-1.10-1
yum install -y compat-libstdc++-33-3.2.3-69.el6 for x86_64
yum install -y compat-libstdc++-33-3.2.3-69.el6 for i686
yum install -y gcc-4.4.4-13.el6
yum install -y gcc-c++-4.4.4-13.el6
yum install -y glibc-2.12-1.7.el6 for x86_64
yum install -y glibc-2.12-1.7.el6 for i686
yum install -y glibc-devel-2.12-1.7.el6 for i686
yum install -y libaio-0.3.107-10.el6
yum install -y libaio-devel-0.3.107-10.el6
yum install -y libgcc-4.4.4-13.el6
yum install -y libstdc++-4.4.4-13.el6 for x86_64
yum install -y libstdc++-4.4.4-13.el6 for i686
yum install -y libstdc++-devel-4.4.4-13.el6
yum install -y libXext for i686
yum install -y libXtst for i686
yum install -y openmotif-2.2.3 for x86_64
yum install -y openmotif22-2.2.3 for x86_64
yum install -y sysstat-9.0.4-11.el6
yum install -y redhat-lsb-4.0-3.el6 for x86_64
yum install -y redhat-lsb-core-4.0-3.el6 for x86_64

References: Oracle Support Doc ID 1298279.1