Importing subscription into MySQL
Last Post 22 Sep 2014 09:36 AM by arun. 4 Replies.
Author Messages Not Resolved
arunUser is Offline
New Member
New Member
Posts:4


--
08 Aug 2014 10:43 AM
    Hello.

    Apelon version '4.1.0.377' installed on a Linux server with MySQL version '5.6' as the database. Schema creation using 'kbcreate.sh' succeeds. But when I attempt to import 'ICD-9-CM_Full.zip' subscription file it fails. The import wizard starts and brings up the GUI where I provide the directory where the ZIP file is present. But when I click the next button I see the first error message which has
    .
    . Table 'dts4.DTS_IMPORT_STATUS' doesn't exist
    .
    But upon clicking next I get to the import mappings. When I start the import I get the message which has
    . Table 'dts4.DTS_NAMESPACE' doesn't exist
    .
    At first look it seems the import is qualifying/prefixing SQL statements with the database/schema name. Is this correct for MySQL? In any case, can anyone help me resolve this? What am I missing?
    .
    .
    Following is a larger subset of logs as relevant as I can make out.
    .
    .
    .
    2014-08-08 11:12:14.063 WARN apelon.data.db [AWT-EventQueue-0] com.apelon.apps.dts.kbimport.gui.CommonNamespaceWizardPanel.setupNamespaceTableData() - Problem reading resume status from target database!, Resume feature will be disabled for [name=ICD-9-CM][type=full]: Error=Table 'dts4.DTS_IMPORT_STATUS' doesn't exist
    2014-08-08 11:12:14.064 WARN apelon.data.db [AWT-EventQueue-0] com.apelon.apps.dts.kbimport.gui.CommonNamespaceWizardPanel.setupNamespaceTableData() - Problem reading resume status from target database!, Resume feature will be disabled for [name=LOINC][type=full]: Error=Table 'dts4.DTS_IMPORT_STATUS' doesn't exist
    2014-08-08 11:12:14.066 WARN apelon.data.db [AWT-EventQueue-0] com.apelon.apps.dts.kbimport.gui.CommonNamespaceWizardPanel.setupNamespaceTableData() - Problem reading resume status from target database!, Resume feature will be disabled for [name=RxNorm R][type=full]: Error=Table 'dts4.DTS_IMPORT_STATUS' doesn't exist
    2014-08-08 11:12:14.067 WARN apelon.data.db [AWT-EventQueue-0] com.apelon.apps.dts.kbimport.gui.CommonNamespaceWizardPanel.setupNamespaceTableData() - Problem reading resume status from target database!, Resume feature will be disabled for [name=SNOMED CT][type=full]: Error=Table 'dts4.DTS_IMPORT_STATUS' doesn't exist
    com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'dts4.DTS_NAMESPACE' doesn't exist
    There may have been a problem querying the database when selecting from the serialized_classifier field in table DTS_CLASSIFIER_GRAPH. Or theremay have been an error when looking for the new classifier graph file in the subscription import data.
    There may have been a problem querying the database when selecting from the serialized_classifier field in table DTS_CLASSIFIER_GRAPH. Or theremay have been an error when looking for the new classifier graph file in the subscription import data.
    Pre-import Classifier Graph Status: No problems with graph version, moving forward with import.
    Pre-import Classifier Graph Status: No problems with graph version, moving forward with import.
    Default Logging Loaded from a file.
    Action: Importing Tables..
    Executing pre import processes :
    Process #1 [SET unique_checks=0]
    Process #2 [SET foreign_key_checks=0]
    Process #3 [SET SESSION tx_isolation='READ-UNCOMMITTED']
    Process #4 [SET sql_log_bin=0]
    Problem getting namespace [ICD-9-CM]Table 'dts4.DTS_NAMESPACE' doesn't exist
    <<<<<<< Performing Checksum checks on content files >>>>>>>
    <<<<<<< All Checksum checks OK. >>>>>>>
    #####
    ##### Loading namespace (ICD-9-CM)
    #####
    ---- Importing into table DTS_AUTHORITY
    *-* Problem retrieving number of records from table.

    * Working on namespace [ICD-9-CM], with property add=true [category=full]
    Problem getting namespace [ICD-9-CM]Table 'dts4.DTS_NAMESPACE' doesn't exist
    BaseTable: import
    com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'dts4.DTS_IMPORT_STATUS' doesn't exist
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
    at com.mysql.jdbc.Util.getInstance(Util.java:386)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1054)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4120)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4052)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2503)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2815)
    at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
    at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2322)
    at com.apelon.dts.db.admin.data.handler.ImportStatusMgr.getLastTableImported(ImportStatusMgr.java:210)
    at com.apelon.dts.db.admin.data.handler.ImportStatusMgr.getLastTableName(ImportStatusMgr.java:60)
    at com.apelon.dts.db.admin.ResumeController.getLastTableImported(ResumeController.java:206)
    at com.apelon.dts.db.admin.ResumeController.resumeTable(ResumeController.java:242)
    at com.apelon.dts.db.admin.data.handler.TableImporter.importTable(TableImporter.java:331)
    at com.apelon.dts.db.admin.DTSBaseTable.processTableImport(DTSBaseTable.java:2354)
    at com.apelon.dts.db.admin.DTSBaseTable.processFullImport(DTSBaseTable.java:1854)
    at com.apelon.dts.db.admin.DTSBaseTable.processImport(DTSBaseTable.java:1833)
    at com.apelon.dts.db.admin.DTSBaseTable.importTable(DTSBaseTable.java:3123)
    at com.apelon.dts.db.admin.DTSBaseTable.importData(DTSBaseTable.java:3081)
    at com.apelon.dts.db.admin.impl.dbcontentmgr.importtables.ImportTables.importTables(ImportTables.java:28)
    at com.apelon.dts.db.admin.impl.dbcontentmgr.importtables.ImportTables.importTables(ImportTables.java:15)
    at com.apelon.dts.db.admin.DbContentMgr.importTables(DbContentMgr.java:546)
    at com.apelon.dts.db.admin.DbContentMgr.importTables(DbContentMgr.java:566)
    at com.apelon.dts.db.admin.DbContentMgr.run(DbContentMgr.java:428)
    at com.apelon.apps.dts.kbimport.gui.progressbar.KBImportTask$ActualTask.importNamespace(KBImportTask.java:361)
    at com.apelon.apps.dts.kbimport.gui.progressbar.KBImportTask$ActualTask.<init>(KBImportTask.java:161)
    at com.apelon.apps.dts.kbimport.gui.progressbar.KBImportTask$1.construct(KBImportTask.java:113)
    at com.apelon.apps.dts.kbimport.gui.progressbar.SwingWorker$2.run(SwingWorker.java:168)
    at java.lang.Thread.run(Thread.java:744)
    Exception running DbContentMgr
    java.sql.SQLException: unable import data: Table 'dts4.DTS_IMPORT_STATUS' doesn't exist
    at com.apelon.dts.db.admin.DTSBaseTable.importData(DTSBaseTable.java:3092)
    at com.apelon.dts.db.admin.impl.dbcontentmgr.importtables.ImportTables.importTables(ImportTables.java:28)
    at com.apelon.dts.db.admin.impl.dbcontentmgr.importtables.ImportTables.importTables(ImportTables.java:15)
    at com.apelon.dts.db.admin.DbContentMgr.importTables(DbContentMgr.java:546)
    at com.apelon.dts.db.admin.DbContentMgr.importTables(DbContentMgr.java:566)
    at com.apelon.dts.db.admin.DbContentMgr.run(DbContentMgr.java:428)
    at com.apelon.apps.dts.kbimport.gui.progressbar.KBImportTask$ActualTask.importNamespace(KBImportTask.java:361)
    arunUser is Offline
    New Member
    New Member
    Posts:4


    --
    21 Sep 2014 08:10 AM
    If this is of help to anyone, the problem really was MySQL's default case sensitivity. Per the Apelon's MySQL specific installation doc the 'my.cnf' (in /etc/mysql on Ubuntu) should have "lower_case_table_names=1" before restarting it and proceeding with Apelon subscription import.

    arunUser is Offline
    New Member
    New Member
    Posts:4


    --
    22 Sep 2014 04:25 AM
    I am attempting to import only RxNORM and am seeing the following error. Any way to get past this?

    The same exception is reported even if I choose only ICD9, or SNOMED-CT, or LOINC, or a all of them together.



    2014-09-22 05:20:15.343 INFO apelon.data.db [Thread-3] com.apelon.dts.db.admin.DTSBaseTable.importData() - ---- Importing into table DTS_NAMESPACE
    2014-09-22 05:20:15.344 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.DTSBaseTable.kbcontentDropConstraints() - numSourceFileLines = 1.0
    2014-09-22 05:20:15.344 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.DTSBaseTable.kbcontentDropConstraints() - targetTableSize = 0.0
    2014-09-22 05:20:15.344 INFO apelon.data.db [Thread-3] com.apelon.dts.db.admin.DTSBaseTable.processTableImport() -
    * Working on namespace [RxNorm R], with property add=true [category=full]
    2014-09-22 05:20:15.345 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Retreiving Table Header Information :
    2014-09-22 05:20:15.346 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - GID with type number
    2014-09-22 05:20:15.346 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - ID with type number
    2014-09-22 05:20:15.346 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - CODE with type string
    2014-09-22 05:20:15.346 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - NAME with type string
    2014-09-22 05:20:15.346 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - NAMESPACE_ID with type number
    2014-09-22 05:20:15.346 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - RELEASE_DATE with type date
    2014-09-22 05:20:15.347 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Retreiving Table Header Information :
    2014-09-22 05:20:15.348 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - GID with type number
    2014-09-22 05:20:15.348 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - ID with type number
    2014-09-22 05:20:15.348 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - CODE with type string
    2014-09-22 05:20:15.348 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - NAME with type string
    2014-09-22 05:20:15.348 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - NAMESPACE_ID with type number
    2014-09-22 05:20:15.348 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - RELEASE_DATE with type date
    2014-09-22 05:20:15.349 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.ResumeController.initStatusMgr() - ---- Initializing import status manager...
    2014-09-22 05:20:15.349 DEBUG apelon.data.db [Thread-3] com.apelon.common.sql.SQL.checkTableExists() - SELECT count(*) AS value FROM information_schema.TABLES WHERE table_schema = SCHEMA() and UPPER(table_name) = UPPER('DTS_IMPORT_STATUS')
    2014-09-22 05:20:15.350 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.DTSBaseTable.getBaseTableName() - No base table found for DTS_NAMESPACE
    2014-09-22 05:20:15.350 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.DTSBaseTable.processTableImport() - importCategory=full
    2014-09-22 05:20:15.350 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.DTSBaseTable.processTableImport() - indexConstrDropped=false
    2014-09-22 05:20:15.350 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.DTSBaseTable.getImportStatement() - Using table 'DTS_NAMESPACE' import statement [insert into dts_namespace (id, name, code, referenced_by, authority_id, is_local, is_writeable, has_semantic_types
    , namespace_type, base_namespace_id,action) values (?,?,?,?,?,?,?,?,?,?,'S')]
    2014-09-22 05:20:15.350 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.DTSBaseTable.getBaseTableImportStatement() - No base table import statement found for DTS_NAMESPACE
    2014-09-22 05:20:15.351 INFO apelon.data.db [Thread-3] com.apelon.dts.db.admin.ResumeController.getLastTableImported() - #### Found last table import [DTS_NAMESPACE] for version [2008.AA.b] and update type [full]
    2014-09-22 05:20:15.354 INFO apelon.data.db [Thread-3] com.apelon.dts.db.admin.ResumeController.isThisTableFullyImported() - #### Table [DTS_NAMESPACE, importCategory=full] isTableFullyImported =false for version [2008.AA.b] and update type [full]
    2014-09-22 05:20:15.355 INFO apelon.data.db [Thread-3] com.apelon.dts.db.admin.ResumeController.getLastImportLineNumber() - #### Found last import line [0] for version [2008.AA.b] and update type [full]
    2014-09-22 05:20:15.364 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.ImportStatusMgr.updateStatusWithTableInfo() - Updated [1] of import status table
    2014-09-22 05:20:15.370 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Retreiving Table Header Information :
    2014-09-22 05:20:15.370 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - ID with type number
    2014-09-22 05:20:15.370 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - NAME with type string
    2014-09-22 05:20:15.371 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - CODE with type string
    2014-09-22 05:20:15.371 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - REFERENCED_BY with type string
    2014-09-22 05:20:15.371 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - AUTHORITY_ID with type number
    2014-09-22 05:20:15.371 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - IS_LOCAL with type string
    2014-09-22 05:20:15.371 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - IS_WRITEABLE with type string
    2014-09-22 05:20:15.371 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - HAS_SEMANTIC_TYPES with type string
    2014-09-22 05:20:15.371 DEBUG apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.parseHeader() - Found header - NAMESPACE_TYPE with type string
    2014-09-22 05:20:15.373 ERROR apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.logError() - Error occured (java.sql.SQLException) while importing line #[2] of file /home/ad/local/apelon/imports/data/1552/DTS_NAMESPACE.full/
    java.sql.SQLException: No value specified for parameter 10
    java.sql.SQLException: No value specified for parameter 10
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975)
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920)
    at com.mysql.jdbc.PreparedStatement.checkAllParametersSet(PreparedStatement.java:2611)
    at com.mysql.jdbc.PreparedStatement.addBatch(PreparedStatement.java:1032)
    at com.apelon.dts.db.admin.data.handler.TableImporter.importRow(TableImporter.java:656)
    at com.apelon.dts.db.admin.data.handler.TableImporter.importTable(TableImporter.java:449)
    at com.apelon.dts.db.admin.DTSBaseTable.processTableImport(DTSBaseTable.java:2354)
    at com.apelon.dts.db.admin.DTSBaseTable.processFullImport(DTSBaseTable.java:1854)
    at com.apelon.dts.db.admin.DTSBaseTable.processImport(DTSBaseTable.java:1833)
    at com.apelon.dts.db.admin.DTSBaseTable.importTable(DTSBaseTable.java:3123)
    at com.apelon.dts.db.admin.DTSBaseTable.importData(DTSBaseTable.java:3081)
    at com.apelon.dts.db.admin.impl.dbcontentmgr.importtables.ImportTables.importTables(ImportTables.java:28)
    at com.apelon.dts.db.admin.impl.dbcontentmgr.importtables.ImportTables.importTables(ImportTables.java:15)
    at com.apelon.dts.db.admin.DbContentMgr.importTables(DbContentMgr.java:546)
    at com.apelon.dts.db.admin.DbContentMgr.importTables(DbContentMgr.java:566)
    at com.apelon.dts.db.admin.DbContentMgr.run(DbContentMgr.java:428)
    at com.apelon.apps.dts.kbimport.gui.progressbar.KBImportTask$ActualTask.importNamespace(KBImportTask.java:361)
    at com.apelon.apps.dts.kbimport.gui.progressbar.KBImportTask$ActualTask.<init>(KBImportTask.java:161)
    at com.apelon.apps.dts.kbimport.gui.progressbar.KBImportTask$1.construct(KBImportTask.java:113)
    at com.apelon.apps.dts.kbimport.gui.progressbar.SwingWorker$2.run(SwingWorker.java:168)
    at java.lang.Thread.run(Thread.java:744)
    2014-09-22 05:20:15.376 INFO apelon.data.db [Thread-3] com.apelon.dts.db.admin.data.handler.TableImporter.rollbackTransaction() - Rollback on last transaction as exception handled.

    jbowieUser is Offline
    Basic Member
    Basic Member
    Posts:110


    --
    22 Sep 2014 09:01 AM
    Could you please check if you are attempting to import a DTS 3.5.2 content file? These files cannot be imported into DTS 4.x. You will need to use the 4.x versions.
    arunUser is Offline
    New Member
    New Member
    Posts:4


    --
    22 Sep 2014 09:36 AM
    The files were indeed incompatible. After using newer files the import complete. Thank you!


    ---