Make sure there is a symbolic link from your modem device to /dev/modem.
You may also have to adjust $X11BASE}/lib/X11/xtel/xtel.lignes for your
modem.
You also have to add:
xtel    1313/tcp
to /etc/services and
xtel            stream  tcp     nowait  root    $X11BASE}/bin/xteld    xteld
to /etc/inetd.conf