IP2I CMS Wiki

User Tools

Site Tools

Trace: • mongoroc_doc
mongoroc_doc

This is an old revision of the document!

Table of Contents

Documentation Mongo DB

Installation

Serveur

Installer une version recente de mongodb sur MONPC

Creation des utilisateurs

Pour la base 'cool_db' 

use cool_db
db.createUser({
  user: 'ian',
  pwd: 'secretPassword',
  roles: [{ role: 'readWrite', db:'cool_db'}]
})

Puis on modifie /etc/mongod.conf 

   # network interfaces
net:
   port: 27017
   bindIp: 0.0.0.0
security:
   authorization: 'enabled'

On peut ainsi acceder a la base cool_db a distance 

mongo -u ian -p secretPassword MONPC/cool_db

Creation de la base

La creation des table dans la base de donnees se fait par insertion d'objets dans les tables

Pour creer la base elle meme, il suffit de se connecter avec mongo et de taper: 

use MaBase

pour creer la base MaBase

Les tables sont creees par insertion par le script python

Client

Du cote client, seul l'acces au base mongodb est necessaire a travers des scripts python. Sur les distributions basees sur debian il suffit d'installer le paquet python-pymongo 

sudo apt-get install python-pymongo

Acces et modification des configurations des ASICs

Representation des ASICs

MongoRoc object

L'objet MongoRoc offre l'access a la base mongo ou les configurations d'ASICs sont stockees

On cree une instance de l'objet avec la method instance(), pour cela il faut avoir defini la variable d'environnment MGDBLOGIN sous la forme: 

 MGDBLOGIN=userXXX/pwdXXX@hostYYY:27017@MYDB  ou
 userXXX est le usernamce
 pwdXXX est le password
 hostYYY est le pc sur lequel la base est installle
 27017 est le port d'acces
 MYDB est le nom de la base

On peut etablir un tunnel ssh pour acceder une base distante 

 ssh -fN -L 27037:hostYYY:27017 user@proxy

Dans ce cas on aura 

 MGDBLOGIN=userXXX/pwdXXX@localhost:27037@MYDB

Methodes de la classe

mongoroc.pdf

Le script mgroc

Le script mgroc permet d'enumerer les etats de la base disponibles et de telecharger un etat donne dans /dev/shm/mgroc/statename_version.json 

mgroc --states
...
CERN2B_1415 1 upload from oracle FEB_2B_1415_16
DIF_ALGECO 1 Un essai avec les DIFs
DIF_ALGECO 2 150 DIF SDHCAL ajoutees
PULSALYS_DOME 1 State de base des 8 boards GRIC pour PULSALYS
PULSALYS_DOME 2 Un test avec un seuil a 120
...
DIFTEST_DOME 1 State de base de la DIF 5 (24 Asics)
DIFTEST_DOME 2 Seuil 140 ASU DIF 5
...
mgroc --download --state=PULSALYS_DOME --version=2
PULSALYS_DOME 2 48  asics

et 

ls -l /dev/shm/mgroc/
total 104
-rw-r--r-- 1 mirabito cms 104744 Jan  7 15:03 PULSALYS_DOME_2.json

Exemples

Les exemples suivants sont pour un FEB, la methodologie est analogue pour les DIFs et les GRICs

Création d'un state

Dans python (MGDBLOGIN doit être déclaré) 

 import MongoRoc as mg
 s=mg.instance()
 s.createNewState("FEB_B15_LYON") 
 s.addFEB("192.168.10.15",2,"PR2B")
 s.uploadNewState("Un example de state avec un FEB et 2 ASICs")

et le state FEB_B15_LYON version 1 est inséré dans la base

Modification d'un state

Thresholds
 import MongoRoc as mg
 s=mg.instance()
 s.download("FEB_B15_LYON",1)
 s.PR2_ChangeVthTime(495,15*256+10,1)
 s.PR2_ChangeVthTime(499,15*256+10,2)
 s.uploadChanges("FEB_B15_LYON","Threshold changed 495 on PR2_1 and 499 on 2")
6 Bits DAC
 import MongoRoc as mg
 s=mg.instance()
 s.download("FEB_B15_LYON",2)
 cor15_1= [0, 0, 0, 0, 3, 2, 2, 2, 3, 1, 1, 2, 0, 1, 1, 0,-1, 1, 1, -1, 1, 1, 1, 0, 1, 0, 1, 0, -1, 0, 2, 0]
 cor15_2= [0, 0, 0, 0, -2, -3, -3, -2, -2, -2, -2, -3, -2, -2, -2, -2, -2, -2, -2,-2, -2, -2, -3, -3, -2,0, -2, 0, -2, 0, -2, 0]
 s.PR2_Correct6BDac(15*256+10,1,cor15_1)
 s.PR2_Correct6BDac(15*256+10,2,cor15_2)
 s.uploadChanges("FEB_B15_LYON","Correction on 6BDAC on the 2 Asics")
Masks channels
 import MongoRoc as mg
 s=mg.instance()
 s.download("FEB_B15_LYON",3)
 chm1=[0,1,2,3,25,27,29,31]
 for c in chm1:
     s.PR2_Change6BDac(15*256+10,1,c,1)
     s.PR2_ChangeMask(15*256+10,1,c,1)
     s.PR2_Change6BDac(15*256+10,2,c,1)
     s.PR2_ChangeMask(15*256+10,2,c,1)
 s.uploadChanges("FEB_B15_LYON","Mask unused channels ")

et on obtient: 

mgroc --states | grep B15
FEB_B15_LYON 1 Un example de state avec un FEB et 2 ASICs
FEB_B15_LYON 2 Threshold changed 495 on PR2_1 and 499 on 2
FEB_B15_LYON 3 Correction on 6BDAC on the 2 Asics
FEB_B15_LYON 4 Mask unused channels

Utilisation pour le jobcontrol

Les configurations du jobcontrol de zdaq sont des fichiers JSON qui doivent etre accessible sur le reseau, l'utilisation de mongodb est donc naturelle

Structure des fichiers JSON

Description generale

Un fichier de configuration est un dictionnaire avec deux tags:

  1. NAME, le nom de la configuration
  2. HOSTS, un dictionnaire des differents PC utilises avec leur liste de processus associes
{
"NAME":"My_first_conf_1",
"HOSTS":{
   "lyopc1":[p1,p2,p3 ],
    "lyopc1":[p1,p2,p3,p4 ],
    "lyopcbuild":[p1,p2]
   }
}

Chaque processus p est decrit par les attributs suivants:

  1. ARGS, une liste des arguments du programme
  2. ENV, une liste des variables d'environnement a declarer lors de l'execution, la variable WEBPORT determinant le port d'acces a l'application doit etre obligatoirement definie.
  3. NAME, le nom logique du programme dans zdaq
  4. PROGRAM, le chemin complet vers le binaire a executer
  5. PARAMETER, un objet JSON pour acceder aux parametres de l'application (format libre)

Liste des descriptions de processus

Liste des processus

SDCC
    {
      "ARGS": [], 
      "ENV": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", 
        "LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:/opt/dhcal/lib:$LD_LIBRARY_PATH", 
        "WEBPORT=41000"
      ], 
      "NAME": "CCCSERVER", 
      "PARAMETER": {
        "device": "DCCCCC01"         <=== FTDI name
      }, 
      "PROGRAM": "/opt/dhcal/bin/lydaq_daemon_Sdcc"
    }
MDCC
    {
      "ARGS": [], 
      "ENV": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", 
        "DIM_DNS_NODE=lyosdhcal10", 
        "LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:/opt/dhcal/levbdim/lib:/opt/dhcal/lib:/opt/dhcal/DQM4HEP/lib:/opt/dhcal/dim/linux:/opt/dhcal/DQM4HEP/lib:/opt/dhcal/root/lib::/opt/dhcal/lcio/v02-00/lib/:$LD_LIBRARY_PATH", 
        "WEBPORT=42000"
      ], 
      "NAME": "MDCCSERVER", 
      "PARAMETER": {
        "device": "MDCC01",      <== FTDI name
        "spilloff": 1000000,     <== Spill parameters
        "spillon": 20000, 
        "spillregister": 64
      }, 
      "PROGRAM": "/opt/dhcal/bin/lydaq_daemon_Mdcc"
    }
FEB Server
    {
      "ARGS": [], 
      "ENV": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", 
        "LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:/opt/dhcal/levbdim/lib:/opt/dhcal/lib:/opt/dhcal/DQM4HEP/lib:/opt/dhcal/dim/linux:/opt/dhcal/DQM4HEP/lib:/opt/dhcal/root/lib::/opt/dhcal/lcio/v02-00/lib/:$LD_LIBRARY_PATH", 
        "CONFDB_WEB=cmsLyon/RPC_2008@ilcconfdb.ipnl.in2p3.fr", 
        "WEBPORT=40000"
      ], 
      "NAME": "TDCSERVER", 
      "PARAMETER": {
        "publish": "tcp://lyosdhcal11:5556",    <== data publication stream
        "tdc": {
          "db": {                               <== DB description
            "mode": "WEB", 
            "state": "FEB_2B_1415_25"
          }, 
          "network": "192.168.10."              <== FEB network path
        }
      }, 
      "PROGRAM": "/opt/dhcal/bin/lydaq_daemon_FebCms"
    }
DIFManager
    {
      "ARGS": [], 
      "ENV": [
        "PATH=/opt/dhcal/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", 
        "LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:/opt/dhcal/lib:$LD_LIBRARY_PATH", 
        "PYTHONPATH=/opt/dhcal/script/:$PYTHONPATH", 
        "CONFDB_WEB=cmsLyon/RPC_2008@ilcconfdb.ipnl.in2p3.fr", 
        "WEBPORT=40000", 
        "INSTANCE=0"
      ], 
      "NAME": "DIFMANAGER", 
      "PARAMETER": {
        "detectorId": 100,     <== Event builder detector id
        "dif": {
          "db": {              <== DB access description
            "mode": "mongo", 
            "state": "DIFTEST_DOME", 
            "version": 3
          }
        }, 
        "publish": "tcp://lyosdhcal9:5556"    <== data publication port
      }, 
      "PROGRAM": "/opt/dhcal/bin/lydaq_daemon_DIF"
    }
DIF Server
    {
      "ARGS": [], 
      "ENV": [
        "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", 
        "LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:/opt/dhcal/lib:$LD_LIBRARY_PATH", 
        "WEBPORT=40000", 
        "INSTANCE=0"
      ], 
      "NAME": "DIFSERVER", 
      "PARAMETER": {
        "builderAddress": "tcp://lyosdhcal10:5556",    <== data collection stream
        "dbcache": "tcp://lyosdhcal10:5555",           <== DB cache (Oracle) address
        "detectorId": 100
      }, 
      "PROGRAM": "/opt/dhcal/bin/lydaq_daemon_Dif"
    }
DAQ control

{ 

     "ARGS": [], 
      "ENV": [
        "LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:/opt/dhcal/lib:$LD_LIBRARY_PATH", 
        "WEBPORT=45000"
      ], 
      "NAME": "FDAQ", 
      "PARAMETER": {       <== Slow control publication information
        "TCPPort": 5500, 
        "location": "DOME_ALGECO_FEB2019",    <== Compulsary for run start
        "period": 30, 
        "s_ctrlreg": "0x815A1B00"
      }, 
      "PROGRAM": "/opt/dhcal/bin/lydaq_daemon_Control"
    }
Event Builder
    {
      "ARGS": [], 
      "ENV": [
        "LD_LIBRARY_PATH=/opt/dhcal/root/lib:/usr/lib:/usr/local/lib:/opt/dhcal/lib:$LD_LIBRARY_PATH", 
        "WEBPORT=46000"
      ], 
      "NAME": "WRITER", 
      "PARAMETER": {
        "dif": 1, 
        "directory": "/data/NAS/TEST_BAKELITE",    <== data writing directory 
        "processor": [                             <==  List of data processor
          "binarywriter", 
          "lydaq_pluggins_monitor", 
          "lydaq_pluggins_rootwriter", 
          "lydaq_pluggins_lciowriter"
        ], 
        "shmpath": "/dev/shm/monitor",             <== Monitoring storage directory
        "stream": [                                <== Liste of listening ports 
          "tcp://*:5556"
        ]
      }, 
      "PROGRAM": "/opt/dhcal/bin/lydaq_daemon_Builder"
    }
Oracle Database Cache
    {
      "ARGS": [], 
      "ENV": [
        "LD_LIBRARY_PATH=/usr/lib:/usr/local/lib:/opt/dhcal/lib:$LD_LIBRARY_PATH", 
        "CONFDB=...",   <= See /etc/ljc.conf for value
        "CONFDB_WEB=cmsLyon/RPC_2008@ilcconfdb.ipnl.in2p3.fr", 
        "WEBPORT=44000"
      ], 
      "NAME": "DBSERVER", 
      "PARAMETER": {
        "mode": "DB", 
        "path": "/dev/shm/", 
        "publish": "tcp://*:5555",   <= publication port to DIF server application
        "state": "Dome_42chambres_Reference_v4_241"
      }, 
      "PROGRAM": "/opt/dhcal/bin/lydaq_daemon_ConfigurationDB"
    }
Zup
    {
      "ARGS": [], 
      "ENV": [
        "LD_LIBRARY_PATH=/opt/dhcal/root/lib:/usr/lib:/usr/local/lib:/opt/dhcal/lib:$LD_LIBRARY_PATH", 
        "WEBPORT=31000"
      ], 
      "NAME": "ZUP", 
      "PARAMETER": {
        "TCPPort": 5500,              <= Slow control publication port
        "location": "SDHCAL_H2SEPT2017", 
        "device": "/dev/ttyUSB0",     <= USB port
        "port": 1                      <= ZUP address
      }
BMP
    {
      "ARGS": [], 
      "ENV": [
        "LD_LIBRARY_PATH=/opt/dhcal/root/lib:/usr/lib:/usr/local/lib:/opt/dhcal/lib:$LD_LIBRARY_PATH", 
        "WEBPORT=32000"
      ], 
      "NAME": "BMP", 
      "PARAMETER": {
        "TCPPort": 5600,    <= slow control publication port
        "location": "SDHCAL_H2SEPT2017"
      }, 
      "PROGRAM": "/opt/dhcal/bin/lydaq_daemon_Bmp"
    }
WIENER
    {
      "ARGS": [], 
      "ENV": [
        "LD_LIBRARY_PATH=/opt/dhcal/root/lib:/usr/lib:/usr/local/lib:/opt/dhcal/lib:$LD_LIBRARY_PATH", 
        "WEBPORT=31000"
      ], 
      "NAME": "ISEG", 
      "PARAMETER": {
        "TCPPort": 5600,          <== Slow control publication port 
        "address": "lyoilchv01",  <== Crate address
        "first": 0,               <== First and last channel
        "last": 55, 
        "location": "SDHCAL_H2SEPT2017"
      }, 
      "PROGRAM": "/opt/dhcal/bin/lydaq_daemon_Wiener"
    }
mongoroc_doc.1578556892.txt.gz · Last modified: 2021/07/10 23:20 (external edit)

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki