HPC/Applications/comsol: Difference between revisions

From CNM Wiki
Jump to navigation Jump to search
 
(108 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Introduction ==
== Introduction ==
<!-- thumb|Comsol features licensed at the CNM -->
<!-- thumb|Comsol features licensed at the CNM -->
[[Image:HPC Comsol licensed modules.png|right|450px|thumb]]
[[Image:HPC Comsol licensed modules.png|right|450px|thumb|Products licensed as of 2021-12.]]
The [http://www.comsol.com/products/multiphysics/ COMSOL Multiphysics] product is available for Mac, Windows, and Linux.
The [http://www.comsol.com/products/multiphysics/ COMSOL Multiphysics] product is available for Mac, Windows, and Linux.


Access to COMSOL is subject to a license agreement and is granted on request to individual users.
== Access ==
* To request access, send a request to [mailto:[email protected]?Subject=Access%20to%20COMSOL [email protected]]. The license agreement restricts access to Argonne employees.
=== COMSOL Multiphysics ===
* Of the [http://www.comsol.com/products/multiphysics/ available features] we have licensed a subset, shown at right.
CNM's license for
* For the native desktop version, contact me ([[User:Stern|stern]]) for installation media. After installation, get [http://www.comsol.com/support/updates/ Product Updates].
[http://www.comsol.com/products/multiphysics/ '''COMSOL Multiphysics''']
(the main graphical user interface application with batch computing capabilities)
is available
'''''upon request'''''&thinsp;
to
'''''Argonne employees'''''&thinsp;
in the following roles:
* ''CNM Facility Users'', under an active proposal with COMSOL in scope, or
* ''NST staff'' (including PostDocs, joint appointments, and students), under intra-divisional discretionary access.
For access, [mailto:helpt@anl.gov?Subject=Access%20to%20COMSOL%20at%20CNM '''submit a support request'''].


There are several ways of running COMSOL
* Of the available features, we have licensed a subset, shown at right.
# Run on Carbon, display on your desktop using X11.
* To have a native desktop version installed on your Argonne-owned computer (laptop or desktop), please contact [mailto:[email protected]?Subject=COMSOL%20license%20access%20request CNM support]. Make sure to review [http://www.comsol.com/support/updates/ Product Updates].
* To run COMSOL on Carbon itself, as GUI or as batch computing jobs, your proposal must have Carbon access in scope.
 
=== COMSOL Server ===
CNM Facility Users who are not Argonne employees are not eligible to access the full '''COMSOL Multiphysics''' product,
in compliance with its licensing terms and conditions.
However, the [https://www.comsol.com/comsol-server '''COMSOL Server'''] product offers an alternative but more narrow means of access.
 
All users, including '''''non-Argonne users''''' may be able to run, over a web interface in their browser,
pre-parameterized "fixed-model" simulations that were made with the '''COMSOL Application Builder''' (requires Windows).
 
To inquire about using COMSOL server applications,
please contact our [mailto:[email protected]?Subject=COMSOL%20server%20usage Support].
We will review possible purchase paths for COMSOL Server, and the ''considerable effort'' likely required by users and the CNM to develop and deploy a COMSOL application.
 
== Support ==
* For questions on how to run or start COMSOL:
*: Submit an [[HPC/Support | '''IT support request''']]
* For questions related to most COMSOL data or scripts, contact COMSOL support at:
*: [mailto:[email protected] '''support'''@comsol.com]
* '''Exception:''' If your support question might contain data that is subject to '''US Export Control''' regulations, contact instead:
*: [mailto:[email protected] '''us-export-control'''@'''us'''.comsol.com]
 
== The modes to run COMSOL ==
There are several ways of running COMSOL.
 
=== Running the desktop application ===
To edit models and analyze results, use the COMSOL graphical user interface (GUI). You can run it in several ways, each explained in more detail in the following sections.
 
# Run on Carbon, display on your desktop using the [http://en.wikipedia.org/wiki/X_Window_System X Window system (X11)].
#* Requires an ''X11 Server'' application running on your client machine, and SSH X11 forwarding.
#* The X11 protocol can be slow over LAN.
# Run on Carbon within a [[HPC/Network Access/Virtual Desktop | virtual desktop environment (VNC)]].
# Run on Carbon within a [[HPC/Network Access/Virtual Desktop | virtual desktop environment (VNC)]].
# Run on your own computer, borrowing Carbon's license.
#* Requires a VNC client on your machine.
# Running a batch calculation on Carbon.
#* It can be confusing where VNC and the app runs.
#* The virtual desktop presents a (deliberatly) sparse windowing environment – Copy&Paste can be difficult to use.
# Run on your own computer, remotely checking out Carbon's license.
#* Requires a full installation of COMSOL on your desktoo or laptop computer.
#* You will require access to the COMSOL license server running on Carbon, either through VPN (fairly easy) or SSH tunneling (can be intimidating for first-time users). See below for details.
 
=== Running a batch calculation ===
When the computational stage of models takes longer than a few minutes to complete, it will be beneficial to submit save the model file and run it in parallel on Carbon.
See the [[#Batch jobs|section below]].


== Use with X11 ==
== Use with X11 ==
Line 28: Line 76:
See general  [[HPC/Network Access/Virtual Desktop | instructions on using VNC]], and the COMSOL example therein.
See general  [[HPC/Network Access/Virtual Desktop | instructions on using VNC]], and the COMSOL example therein.


== Running a native desktop version ==
== Running native desktop version ==
To get the fastest response from the COMSOL GUI, run a native COMSOL version on your desktop machine or laptop, called ''client machine'' below.


=== Direct license server connection - CNM/NST networks only ===
For client machines inside the CNM, follow the next section.
Configure COMSOL to use divisional license servers. Edit <code><COMSOLDIR>/license/license.dat</code> and enter:
For a client machine outside of CNM networks, configure it to check out a license from the Comsol license server running on Carbon, using one of two ways. Choose one that you are more comfortable with.
SERVER cmgmt3 ANY 1718
SERVER cmgmt4 ANY 1718
SERVER csched1 ANY 1718
USE_SERVER
; Note: These servers are reachable only from within the CNM.


=== Use license forwarding - Other networks ===
Enter license details either in the COMSOL Setup configuration dialog or later in COMSOL's license file at <code>&lt;COMSOLDIR&gt;/license/license.dat</code>, where <code>&lt;COMSOLDIR&gt;</code> is located by default at:
<!-- [[Image:HPC Comsol license tunneling for PuTTY.png|thumb|Example for Comsol license tunneling using SSH under the popular PuTTY application.
* '''Windows''': C:\Program files\COMSOL\COMSOL5xx
Adapt these settings for your SSH applications.|350px]]  -->
* '''Linux''': /usr/local/comsol5xx
To get the fastest response from the COMSOL GUI, run a native COMSOL version on your desktop and configure it to check out a license from the Comsol license server running on Carbon.
* '''Mac''': /Applications/COMSOL5xx
("5xx" will be your installed version)


There are actually two license servers on Carbon, a primary and a secondary which can take over as needed. Unfortunately, the automatic server selection only works for nodes within Carbon, and fails when using a native desktop version.
=== Inside CNM or over VPN from outside ===
Use the following steps to set up license checkout.
==== Ensure correct DNS search domains ====


==== Configure port forwarding ====
Make sure that <code>cnm.anl.gov</code> is listed in the DNS Search Domains for your current network profile.
* On '''your desktop machine''', set up SSH tunneling. This step is only needed once.
* If the client machine is directly in CNM networks, this should be the default.
** Windows: [[HPC/Network Access/PuTTY Configuration/Accessing Carbon licenses remotely | Configure PuTTY ]]
* If the client machine is connected over VPN, edit the respective network profile.
** Linux/Mac/Cygwin: edit the file <code>~/.ssh/config</code> and add to the <code>Host clogin</code> section:
:* On a Mac, find the VPN configuration details under System Preferences, Network, Advanced… . You may need to unlock the Network panel first before you can make changes under "Advanced…". It appears that those changes are being saved and will be activated automatically the next time that this VPN session gets started.
: [[Image:VNC_add_cnm_to_DNS_1.png|400px]] [[Image:VNC_add_cnm_to_DNS_2.png|400px]]
:* On Windows or Linux machines, take ''analogous'' steps, using the platform's own configuration tools.
 
==== Configure License servers ====
* On your client machine, use the following entries for the COMSOL license file, or enter the respective host names and ports into the COMSOL Setup license configuration dialog:
<source lang="make">
<source lang="make">
Host clogin
SERVER clicense1 ANY 1718
SERVER clicense2 ANY 1718
# COMSOL licensing
SERVER clicense3 ANY 1718
# Initial ("negotiation") ports of FlexLM's lmgrd; may be forwarded to different local port numbers.
USE_SERVER
LocalForward 33318 mgmt03:1718
LocalForward 33418 mgmt04:1718
# Secondary ("payload") ports of the COMSOL vendor daemon; local and remote port numbers must match.
LocalForward  1719 mgmt03:1719
LocalForward  1720 mgmt04:1720
</source>
</source>
: Note …'''3'''18 vs. …'''4'''18 in the the first line pair.
: '''Note:''' Use ''short host names'' as given here. (Technical Reason: The license must be usable in both Carbon's internal network as well as networks outside of Carbon).
* Stop any previous connection that you might have to clogin.
* Start COMSOL.
* Connect to clogin. This will start the port forwarding configured in the previous step.
** On PuTTY, open the Clogin session.
** On Linux/Mac/Cygwin, type:
ssh -v clogin


==== Configure COMSOL to use forwarded ports ====
==== Network Troubleshooting ====
* Start the Comsol installer on your machine. When asked for licensing use the following type:
When using the license access model descibed here (as opposed to tunneled over SSH), ensure the following:
** License type: networked server
* Verify that short host names can be resolved to IP addresses. Either one of the following commands should produce sensible output:
* For License hosts use ''one'' of the following entries. If the first failed, use the second.
host clicense1
** localhost, port 33318
: or:
** localhost, port 33418
ping -c 3 clicense1
* Complete the installer.
* Verify that ''both license ports on the same server'' can be reached from your client machine:
* Locate and start the COMSOL Multiphysics application.
telnet '''clicense1''' 1718
telnet '''clicense1''' 1719
: or:
telnet '''clicense2''' 1718
telnet '''clicense2''' 1719
: A ''successful'' connection looks like:
Trying 146.139.x.y...
Connected to clicense1.cnm.anl.gov.
Escape character is '^]'.
:: Do not enter any data. Instead,  close each connection by hitting <code>Ctrl</code><code>]</code> (closing square bracket), followed by <code>Ctrl</code><code>D</code>.
: A ''failed'' connection gives one of these errors:
telnet: connect to address 146.139.x.y: Operation timed out
: or:
telnet: connect to address 146.139.x.y: Connection refused
:: In these cases, ask that your IP firewall configuration be reviewed and changed.


==== Preparing to run COMSOL ====
=== Outside CNM, using SSH Tunneling ===
* Have an SSH connection to clogin active. It provides the forwarded ports to your machine.
If you're comfortable with ssh and tunneling, do the following on your client machine:
* Then start the COMSOL application normally.
* Enter into the COMSOL license file:
 
==== Addendum: Point to an alternate license server ====
The COMSOL application determines the license server to contact by the file <code><COMSOLDIR>/license/license.dat</code>. The file should look as follows:
<source lang="make">
<source lang="make">
SERVER localhost ANY 33318
SERVER 127.0.0.1 ANY 1718
SERVER localhost ANY 33418
USE_SERVER
USE_SERVER
</source>
</source>
Most of the time, the first license server is active and will respond locally on port 33318.
: '''Note:''' Do not use "localhost", as this will be directly (and incorrectly) interpreted by COMSOL to mean that the current machine actually ''is'' a license server.
If that port does not respond, COMSOL fails with an error.
Ideally, COMSOL should contact the alternate server port as given by the second SERVER line, but in COMSOL43a that line is ignored.


To temporarily use the alternate server, comment out the first SERVER line from the <code>license.dat</code> file:
Connect to mega or a Carbon login node with one of the following ssh commands:
<source lang="make">
* First, set up a tunnel to the primary license server:
#SERVER localhost ANY 33318
ssh  -L 1718:'''clicense1''':1718  -L 1719:'''clicense1''':1719  mega
SERVER localhost ANY 33319
* Start COMSOL.
USE_SERVER
: If you get an error that the number of licenses is exhausted, that is at least confirmation that you correctly reached the license server.
</source>
 
Then restart COMSOL. You do not need to change the SSH configuration or re-open the connection to clogin.
If the above attempt takes a long time or hangs entirely, it may be because the license service currently runs on the secondary server.
* Connect to the alternate license server:
ssh  -L 1718:'''clicense2''':1718  -L 1719:'''clicense2''':1719  mega
* Start COMSOL.
: Note: With SSH tunneling, an automatich selection of an active license server from a redundant set cannot be done.  Instead, the user must try one server first, then the other.
 
== License Troubleshooting ==
The following are typical error messages.
=== Could not obtain license for COMSOL Multiphysics GUI ===
* Error details:
License error: -4.
'''Licensed number of users already reached.'''
Feature:      COMSOLGUI
License path:  /Users/stern/Applications/COMSOL52a/license/license.dat:
FlexNet Licensing error:-4,132
* Reason: The connection to the license server succeeded, but the license token(s) you requested are in use.
* Solution: Wait a bit, or [http://carbonmgmt/ganglia/lic.php identify who's using COMSOL] and politely ask the user when they might be finished with their work.
 
=== License server machine is down or not responding ===
* Error details:
  License error: -96.
  See the system administrator about starting the license server system, or
  make sure you're referring to the right host (see LM_LICENSE_FILE).
Feature:      SERIAL
Hostname:      clicense3
License path:  /Users/stern/Applications/COMSOL52a/license/license.dat:
FlexNet Licensing error:-96,7
* Reason: The license servers cannot be reached at all. The <code>ping</code> or <code>host</code> tests mentioned above fail.
* Solution: Review the client license file and DNS lookup domains.
 
=== License file does not support this version ===
* Error details:
  Feature: SERIAL
'''Application version > License version: 5.21 > 3.5'''
License path: /Users/stern/Applications/COMSOL52a/license/license.dat:
FlexNet Licensing error:-21,126
* Reason: Only the first of two steps for connecting to the license servers succeeds, which may happen when the redundant license service runs on the alternate server instead of the one you're connected to.
* Solution: When using SSH, try <code>clicense2</code> instead of  <code>clicense1</code>, or vice versa. Revisit your <code>~/.ssh/config</code> file or use the alternate command line as shown in the SSH section above.


== Multiple CPU cores ==
== Multiple CPU cores ==
Line 123: Line 209:
See:
See:
  $COMSOL_HOME/sample.job
  $COMSOL_HOME/sample.job
== Important COMSOL Support Articles ==
* Toplevel suport page: https://www.comsol.com/support
=== KB Articles ===
* [https://www.comsol.com/support/knowledgebase/1001 Running COMSOL® in parallel on clusters]
* [https://www.comsol.com/support/knowledgebase/1250 Running parametric sweeps, batch sweeps, and cluster sweeps from the command line]
* [https://www.comsol.com/support/knowledgebase/1255 Reducing the amount of solution data stored in a model]
* [https://www.comsol.com/support/knowledgebase/830 Handling large models]
=== Blog-style articles ===
* [https://www.comsol.com/blogs/the-power-of-the-batch-sweep/ The Power of the Batch Sweep]
* [https://www.comsol.com/blogs/how-to-run-simulations-in-batch-mode-from-the-command-line/ How to Run Simulations in Batch Mode from the Command Line]

Latest revision as of 17:37, September 1, 2022

Introduction

Products licensed as of 2021-12.

The COMSOL Multiphysics product is available for Mac, Windows, and Linux.

Access

COMSOL Multiphysics

CNM's license for COMSOL Multiphysics (the main graphical user interface application with batch computing capabilities) is available upon request  to Argonne employees  in the following roles:

  • CNM Facility Users, under an active proposal with COMSOL in scope, or
  • NST staff (including PostDocs, joint appointments, and students), under intra-divisional discretionary access.

For access, submit a support request.

  • Of the available features, we have licensed a subset, shown at right.
  • To have a native desktop version installed on your Argonne-owned computer (laptop or desktop), please contact CNM support. Make sure to review Product Updates.
  • To run COMSOL on Carbon itself, as GUI or as batch computing jobs, your proposal must have Carbon access in scope.

COMSOL Server

CNM Facility Users who are not Argonne employees are not eligible to access the full COMSOL Multiphysics product, in compliance with its licensing terms and conditions. However, the COMSOL Server product offers an alternative but more narrow means of access.

All users, including non-Argonne users may be able to run, over a web interface in their browser, pre-parameterized "fixed-model" simulations that were made with the COMSOL Application Builder (requires Windows).

To inquire about using COMSOL server applications, please contact our Support. We will review possible purchase paths for COMSOL Server, and the considerable effort likely required by users and the CNM to develop and deploy a COMSOL application.

Support

The modes to run COMSOL

There are several ways of running COMSOL.

Running the desktop application

To edit models and analyze results, use the COMSOL graphical user interface (GUI). You can run it in several ways, each explained in more detail in the following sections.

  1. Run on Carbon, display on your desktop using the X Window system (X11).
    • Requires an X11 Server application running on your client machine, and SSH X11 forwarding.
    • The X11 protocol can be slow over LAN.
  2. Run on Carbon within a virtual desktop environment (VNC).
    • Requires a VNC client on your machine.
    • It can be confusing where VNC and the app runs.
    • The virtual desktop presents a (deliberatly) sparse windowing environment – Copy&Paste can be difficult to use.
  3. Run on your own computer, remotely checking out Carbon's license.
    • Requires a full installation of COMSOL on your desktoo or laptop computer.
    • You will require access to the COMSOL license server running on Carbon, either through VPN (fairly easy) or SSH tunneling (can be intimidating for first-time users). See below for details.

Running a batch calculation

When the computational stage of models takes longer than a few minutes to complete, it will be beneficial to submit save the model file and run it in parallel on Carbon. See the section below.

Use with X11

For casual use, and with X11 tunneling to Carbon set up, simply add to your ~/.bashrc file on Carbon:

module load comsol

Then, from the command line on a Carbon login node, start the COMSOL GUI:

comsol &

Comsol will use a number of cores to run calculations multithreaded by default. (On Carbon, COMSOL_NUM_THREADS is set.)

Quit the application as soon as you are done. Only one user can use Comsol at a time.

Use with VNC

See general instructions on using VNC, and the COMSOL example therein.

Running native desktop version

To get the fastest response from the COMSOL GUI, run a native COMSOL version on your desktop machine or laptop, called client machine below.

For client machines inside the CNM, follow the next section. For a client machine outside of CNM networks, configure it to check out a license from the Comsol license server running on Carbon, using one of two ways. Choose one that you are more comfortable with.

Enter license details either in the COMSOL Setup configuration dialog or later in COMSOL's license file at <COMSOLDIR>/license/license.dat, where <COMSOLDIR> is located by default at:

  • Windows: C:\Program files\COMSOL\COMSOL5xx
  • Linux: /usr/local/comsol5xx
  • Mac: /Applications/COMSOL5xx

("5xx" will be your installed version)

Inside CNM or over VPN from outside

Ensure correct DNS search domains

Make sure that cnm.anl.gov is listed in the DNS Search Domains for your current network profile.

  • If the client machine is directly in CNM networks, this should be the default.
  • If the client machine is connected over VPN, edit the respective network profile.
  • On a Mac, find the VPN configuration details under System Preferences, Network, Advanced… . You may need to unlock the Network panel first before you can make changes under "Advanced…". It appears that those changes are being saved and will be activated automatically the next time that this VPN session gets started.
VNC add cnm to DNS 1.png VNC add cnm to DNS 2.png
  • On Windows or Linux machines, take analogous steps, using the platform's own configuration tools.

Configure License servers

  • On your client machine, use the following entries for the COMSOL license file, or enter the respective host names and ports into the COMSOL Setup license configuration dialog:
SERVER clicense1 ANY 1718
SERVER clicense2 ANY 1718
SERVER clicense3 ANY 1718
USE_SERVER
Note: Use short host names as given here. (Technical Reason: The license must be usable in both Carbon's internal network as well as networks outside of Carbon).
  • Start COMSOL.

Network Troubleshooting

When using the license access model descibed here (as opposed to tunneled over SSH), ensure the following:

  • Verify that short host names can be resolved to IP addresses. Either one of the following commands should produce sensible output:
host clicense1
or:
ping -c 3 clicense1
  • Verify that both license ports on the same server can be reached from your client machine:
telnet clicense1 1718
telnet clicense1 1719
or:
telnet clicense2 1718
telnet clicense2 1719
A successful connection looks like:
Trying 146.139.x.y...
Connected to clicense1.cnm.anl.gov.
Escape character is '^]'.
Do not enter any data. Instead, close each connection by hitting Ctrl] (closing square bracket), followed by CtrlD.
A failed connection gives one of these errors:
telnet: connect to address 146.139.x.y: Operation timed out
or:
telnet: connect to address 146.139.x.y: Connection refused
In these cases, ask that your IP firewall configuration be reviewed and changed.

Outside CNM, using SSH Tunneling

If you're comfortable with ssh and tunneling, do the following on your client machine:

  • Enter into the COMSOL license file:
SERVER 127.0.0.1 ANY 1718
USE_SERVER
Note: Do not use "localhost", as this will be directly (and incorrectly) interpreted by COMSOL to mean that the current machine actually is a license server.

Connect to mega or a Carbon login node with one of the following ssh commands:

  • First, set up a tunnel to the primary license server:
ssh  -L 1718:clicense1:1718  -L 1719:clicense1:1719  mega
  • Start COMSOL.
If you get an error that the number of licenses is exhausted, that is at least confirmation that you correctly reached the license server.

If the above attempt takes a long time or hangs entirely, it may be because the license service currently runs on the secondary server.

  • Connect to the alternate license server:
ssh  -L 1718:clicense2:1718  -L 1719:clicense2:1719  mega
  • Start COMSOL.
Note: With SSH tunneling, an automatich selection of an active license server from a redundant set cannot be done. Instead, the user must try one server first, then the other.

License Troubleshooting

The following are typical error messages.

Could not obtain license for COMSOL Multiphysics GUI

  • Error details:
License error: -4.
Licensed number of users already reached.
Feature:       COMSOLGUI
License path:  /Users/stern/Applications/COMSOL52a/license/license.dat:
FlexNet Licensing error:-4,132
  • Reason: The connection to the license server succeeded, but the license token(s) you requested are in use.
  • Solution: Wait a bit, or identify who's using COMSOL and politely ask the user when they might be finished with their work.

License server machine is down or not responding

  • Error details:
 License error: -96.
 See the system administrator about starting the license server system, or
 make sure you're referring to the right host (see LM_LICENSE_FILE).
Feature:       SERIAL
Hostname:      clicense3
License path:  /Users/stern/Applications/COMSOL52a/license/license.dat:
FlexNet Licensing error:-96,7
  • Reason: The license servers cannot be reached at all. The ping or host tests mentioned above fail.
  • Solution: Review the client license file and DNS lookup domains.

License file does not support this version

  • Error details:
 Feature: SERIAL
Application version > License version: 5.21 > 3.5
License path: /Users/stern/Applications/COMSOL52a/license/license.dat:
FlexNet Licensing error:-21,126
  • Reason: Only the first of two steps for connecting to the license servers succeeds, which may happen when the redundant license service runs on the alternate server instead of the one you're connected to.
  • Solution: When using SSH, try clicense2 instead of clicense1, or vice versa. Revisit your ~/.ssh/config file or use the alternate command line as shown in the SSH section above.

Multiple CPU cores

Ways to start multi-core (from http://www.comsol.com/support/knowledgebase/1096/)

comsol -np 8
export COMSOL_NUM_THREADS=8    # automatically provided on Carbon
comsol
  • COMSOL does not benefit from hyperthreading (oversubscribing the physical core count).

Batch jobs

Sample Job file

See:

$COMSOL_HOME/sample.job

Important COMSOL Support Articles

KB Articles

Blog-style articles