Kohei Nozaki's blog 

Entries tagged [james]

Adding indexes to tables of Apache James 3

Posted on Sunday Nov 15, 2015 at 05:06PM in Technology

I’m using Apache James (3.0-beta5-SNAPSHOT) as the mail server for my domain. It’s running with JPA backend on PostgreSQL.

Today I’ve found that some slow queries are logged as the number of emails has been grown through the days. The slowest two are following:

  • SELECT t0.property_id, t0.property_line_number, t0.property_local_name, t0.property_name_space, t0.property_value FROM public.JAMES_MAIL_PROPERTY t0 WHERE t0.mailbox_id = $1 AND t0.mail_uid = $2 ORDER BY t0.property_line_number ASC

  • SELECT t0.userflag_id, t0.userflag_name FROM public.JAMES_MAIL_USERFLAG t0 WHERE t0.mailbox_id = $1 AND t0.mail_uid = $2 ORDER BY t0.userflag_id ASC

These queries are used sequential scan. It’s terrible for thousands of rows so I’ve created a couple of indexes to avoid sequential scan as follows:

  • create index on james_mail_property (mailbox_id, mail_uid);

  • create index on james_mail_userflag (mailbox_id, mail_uid);

They seems work expectedly as I’ve confirmed that now these queries are using index scan instead.

Comments [0]

Tags: james postgresql

Configuring Apache James to use PostgreSQL as the backend

Posted on Sunday Apr 19, 2015 at 09:37PM in Technology

UPDATE: I’m running into a strange problem of using PostgreSQL as backend with Thunderbird. I recommend to use an other database as the backend at the moment. http://www.mail-archive.com/server-user%40james.apache.org/msg14715.html

UPDATE2: I created a patch to solve the issue and it seems fine. for detail and download the patch see: https://issues.apache.org/jira/browse/MAILBOX-228

I have been used Apache James with Embedded Derby, but its CPU/IO consumption is increasing day by day. I’m not sure where is the bottleneck but I decided to move to PostgreSQL as its backend anyway.

  1. Put JDBC driver (I used postgresql-9.3-1100.jdbc41.jar) into $JAMES_HOME/conf/lib

  2. Create james-database.properties from template, and put it into $JAMES_HOME/conf as follows

    database.driverClassName=org.postgresql.Driver
database.url=jdbc:postgresql://127.0.0.1:5432/james
database.username=james
database.password=***

vendorAdapter.database=POSTGRESQL

openjpa.streaming=true

  3. Restart James instance

  4. Create domains, addresses and aliases:

    ./james-cli.sh -h localhost -p 9999 adddomain example.org
./james-cli.sh -h localhost -p 9999 adduser kyle@example.org ***
./james-cli.sh -p 9999 -h localhost addaddressmapping postmaster example.org kyle@example.org

Note

This guide claims that setting standard_conforming_strings=off is needed, but I guess this issue may address this problem, so I don’t use this setting at the moment. my James installation is shipped with openjpa-2.2.1.jar and the issue was fixed in 2.2.0.

Comments [3]

Tags: james postgres

Derby database backup script

Posted on Friday Feb 20, 2015 at 02:49PM in Technology

As Roller, and a virtual machine on VMware Fusion, I wrote an another Ant script which backups an Apache Derby database to automate backup of data of my Apache James server. the script has easy purge function as a target named purge too. intended environment is as follows:

  • Linux server

  • Accepts connection via ssh

  • Has executable ij command which is simple CLI JDBC frontend program shipped with Derby

  • Requires Derby instance to listen a port

It works as follows:

  1. Invoke SYSCS_UTIL.SYSCS_FREEZE_DATABASE() with ij to freeze the database

  2. Create a tarball of the database

  3. Invoke SYSCS_UTIL.SYSCS_UNFREEZE_DATABASE() with ij to unfreeze the database

  4. Download the tarball

  5. Delete the tarball

A bad thing is that due to lack of streaming download in sshexec task, it needs extra free space on the server. the script can be obtained from my GitHub repository.

Comments [0]

Tags: ant backup derby james

Configuring James to sign DKIM for outbound mails

Posted on Saturday Feb 07, 2015 at 01:20PM in Technology

Environment

  • Apache jDKIM 0.3-SNAPSHOT

  • Apache James 3.0.0-beta5-SNAPSHOT

  • Oracle JDK7u51

Obtaining and building jDKIM

svn co http://svn.apache.org/repos/asf/james/jdkim/trunk jdkim
cd jdkim
mvn clean install

Put jars of jDKIM into James

Put following files packed in $JDKIM_HOME/assemble/target/apache-jdkim-0.3-SNAPSHOT-bin.tar.gz into $JAMES_HOME/conf/lib of your James server:

  1. apache-jdkim-library-0.3-SNAPSHOT.jar

  2. apache-jdkim-mailets-0.3-SNAPSHOT.jar

  3. lib/not-yet-commons-ssl-0.3.11.jar

Creating key pair

openssl genrsa -out dkim-private.pem 1024 -outform PEM
openssl rsa -in dkim-private.pem -out dkim-public.pem -pubout -outform PEM

Registering public key into DNS

20150207._domainkey IN TXT "k=rsa; p=[YOUR PUBLIC KEY HERE EXCLUDE THE HEADER AND FOOTER];"

20150207 is a selector and which identifies the key. it’s recommended to rotate periodically so timestamp manner like YYYYMMDD is reasonable. GMail is used this way too.

Following command is useful if you don’t want to concatenate and strip the header and footer by hand:

awk 'NR>1{a[++k]=$0}END{for(i=1;i<k;i++){printf("%s", a[i])}; printf("\n")}' dkim-public.pem

You can test whether the key is successfully registered as follows:

host -t txt 20150207._domainkey.example.com

Defining Mailets

Put following definition to $JAMES_HOME/conf/mailetcontainer.xml. make sure to put the value of s (selector) that is the same as one which you just have registered in DNS. and you have to replace d (domain) with your domain. note that you should put following definition to place just before <mailet match="All" class="RemoteDelivery"> element to prevent altering mails after signing.

<mailet match="All" class="org.apache.james.jdkim.mailets.ConvertTo7Bit"/>

<mailet match="All" class="org.apache.james.jdkim.mailets.DKIMSign">
  <signatureTemplate>v=1; s=20150207; d=example.com; c=relaxed/relaxed; h=Message-ID:Date:Subject:From:To:MIME-Version:Content-Type; a=rsa-sha256; bh=; b=;</signatureTemplate>
  <privateKey>
-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----
  </privateKey>
</mailet>

If you have passphrase in your key, you need to put it here as privateKeyPassword argument.

Testing DKIM signature

Go https://www.mail-tester.com and send a mail from your James server.

Invalid signature on Quoted-Printable mails

It works fine with 7bit plain text mails but when I send a long-line mail with Apple Mail which will be converted to Quoted-Printable, James creates invalid signature. I’m investing this problem now but not unsure at present time. maybe investigation in DKIMSign, CRLFOutputStream, RemoteDelivery and SMTPTransport are needed because they have many code of manipulating content.

UPDATE I found the cause. you need to put following definition inside the mailet named RemoteDelivery. JavaMail automatically converts quoted-printable mails to 8bit plain-text mail because RemoteDelivery class sets it to true at its run() method.

<mail.smtp.allow8bitmime>false</mail.smtp.allow8bitmime>

References

  1. http://james.apache.org/jdkim/mailets/

  2. http://james.apache.org/jdkim/

  3. http://mail-archives.apache.org/mod_mbox/james-server-user/201410.mbox/%3C544FD474.2040906%40malcolms.com%3E

  4. http://www.gettingemaildelivered.com/dkim-explained-how-to-set-up-and-use-domainkeys-identified-mail-effectively

  5. http://www.dataenter.com/doc/general_domainkeys.htm#Generation

  6. https://scottlinux.com/2012/10/27/how-to-fetch-dkim-records-from-dns/

Comments [2]

Tags: dkim james

Configuring James to use SSL

Posted on Saturday Feb 07, 2015 at 12:07AM in Technology

Environment

  • Apache James 3.0.0beta5-SNAPSHOT

  • Oracle JDK8u31

  • CentOS 6.5

Requirement

  • Listening IMAPS at 993

  • Listening SMTPS at 465 (for mail client)

  • Listening SMTP at 25 (for accepting connection from other SMTP server. STARTTLS enabled)

  • Expose these ports with forwarding by iptables

Put Java KeyStore

I put it to $JAMES_HOME/conf/mykeystore.jks. check this posting for how to create the Java KeyStore.

Configuring IMAPS

  1. Change port number in bind element as follows:

    <bind>0.0.0.0:10993</bind>

  2. Edit tls element as follows:

    <tls socketTLS="true" startTLS="false">
 <keystore>file://conf/mykeystore.jks</keystore>
 <secret>PASSPHRASE</secret>
 <provider>org.bouncycastle.jce.provider.BouncyCastleProvider</provider>
</tls>

    I guess I don’t have BouncyCastleProvider class in my classpath but it works.

Configuring SMTPS

  1. Make a whole copy of smtpserver element in smtpserver.xml.

  2. Change jmxName element of second smtpserver element:

    <jmxName>smtpsserver</jmxName>

  3. Change port number in bind element as well:

    <bind>0.0.0.0:10465</bind>

  4. Edit tls element too:

    <tls socketTLS="true" startTLS="false">
 <keystore>file://conf/mykeystore.jks</keystore>
 <secret>PASSPHRASE</secret>
 <provider>org.bouncycastle.jce.provider.BouncyCastleProvider</provider>
 <algorithm>SunX509</algorithm>
</tls>

  5. Also enabling of following configuration is required for some mail client such as Apple Mail:

    <authRequired>announce</authRequired>

Configuring SMTP

Edit tls element inside first smtpserver element. set true to startTLS:

<tls socketTLS="false" startTLS="true">
 <keystore>file://conf/mykeystore.jks</keystore>
 <secret>PASSPHRASE</secret>
 <provider>org.bouncycastle.jce.provider.BouncyCastleProvider</provider>
 <algorithm>SunX509</algorithm>
</tls>

Delete a Mailet from mailetcontainer.xml

There’s a Mailet which needs to get removed when you use SMTP auth. this solves the problem that getting Storing mail …​ in file://var/mail/relay-denied/. so delete following fragment from $JAMES_HOME/conf/mailetcontainer.xml:

<mailet match="RemoteAddrNotInNetwork=127.0.0.1" class="ToProcessor">
  <processor>relay-denied</processor>
  <notice>550 - Requested action not taken: relaying denied</notice>
</mailet>

In my case, a class named AuthRequiredToRelayRcptHook prevents open relay, without that Mailet.

Configuring iptables

An example as follows. write it to /etc/sysconfig/iptables and issue sudo service iptables restart.

*nat
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -i eth0 -p tcp --dport 25 -j DNAT --to-destination :10025
-A PREROUTING -i eth0 -p tcp --dport 465 -j DNAT --to-destination :10465
-A PREROUTING -i eth0 -p tcp --dport 993 -j DNAT --to-destination :10993
COMMIT
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 10025 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 10465 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 10993 -j ACCEPT
COMMIT

Open relay check of SMTP server

Just in case, do it with testing site like http://www.aupads.org/test-relay.html

Comments [1]

Tags: james ssl

Hot tags

Below are the most often used tags in the blog. Hover over a tag to see a count of entries, click a tag to see the most recent posts with the tag.