error message on the one-step workflow: formatdb

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

error message on the one-step workflow: formatdb

gary_xie
We managed to get the java environment setup and get past the earlier  
problem, but now when I create a formatdb pipeline and run it, I get  
this error.

> more replace_config_keys.stderr
Died with 'Couldn't add key $; PIPELINEID $;=22 to section  
[component]' at /opt/ergatis/bin/replace_config_keys.pl line 241


and if I look at the  formatdb.default.user.config
it doesn't contain the key, so it can't replace it.


But, if I look at  ergatis/docs/formatdb.config, it does have the keys  
(with no values):

;the following keys are replaced at runtime by the invocation script
$;COMPONENT_CONFIG$; =
$;COMPONENT_XML$; =
$;PIPELINE_XML$; =
$;PIPELINEID$; =


I would guess it uses the formatdb.config to create the config file  
for the pipeline, but don't understand why I'm losing the keys and  
getting this error.
Can anyone help?
Thanks,
Gary

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

Jason Kwan
Hi Gary,

I had a similar problem with that part part of replace_config_keys.pl
(i.e. the add_keys subroutine).  For me anyway, it turned out that the
version of Config::Inifiles that I had gives an error when you try to do
setval() on a value that doesn't already exist.

So I changed the original sub from:
sub add_keys{
     my($cfg,$section,@keys) = @_;
     $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
     foreach my $kv (@keys){
     my($key,$value) = split(/=/,$kv);
     $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
     my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
     if(!$ret){
         $logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
     }
     }
}

to:
sub add_keys{
     my($cfg,$section,@keys) = @_;
     $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
     foreach my $kv (@keys){
     my($key,$value) = split(/=/,$kv);
     $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
     my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
     if(!$ret){
         $cfg->newval($section, $delimeter.$key.$delimeter,$value);
         #$logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
     }
     }
}

Jason

On 6/7/11 4:11 PM, Gary Xie wrote:

> We managed to get the java environment setup and get past the earlier
> problem, but now when I create a formatdb pipeline and run it, I get
> this error.
>
>> more replace_config_keys.stderr
> Died with 'Couldn't add key $; PIPELINEID $;=22 to section
> [component]' at /opt/ergatis/bin/replace_config_keys.pl line 241
>
>
> and if I look at the  formatdb.default.user.config
> it doesn't contain the key, so it can't replace it.
>
>
> But, if I look at  ergatis/docs/formatdb.config, it does have the keys
> (with no values):
>
> ;the following keys are replaced at runtime by the invocation script
> $;COMPONENT_CONFIG$; =
> $;COMPONENT_XML$; =
> $;PIPELINE_XML$; =
> $;PIPELINEID$; =
>
>
> I would guess it uses the formatdb.config to create the config file
> for the pipeline, but don't understand why I'm losing the keys and
> getting this error.
> Can anyone help?
> Thanks,
> Gary
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

gary_xie
Thanks, Jason,
Here are details  on the error message I got.
Gary

formatted source viewer: /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.default.user.config

[interface]

  • classification = file / indexing

[parameters]

  • enter T for protein, F for nucleotide
  • $;PROTEIN$; = F

[input]

  • space-separated list
  • $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna

[output]

  • $;OUTPUT_TOKEN$; = default
  • $;OUTPUT_DIRECTORY$; = $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • call it whatever you like
  • $;DATABASE_NAME$; = nucdb

[component]

  • $;COMPONENT_NAME$; = formatdb
  • $;DESCRIPTION$; = none
  • $;WORKFLOW_REPOSITORY$; = $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • $;PIPELINE_TOKEN$; = unnamed
  • The version,revision,tag here is set by an interpolated CVS tag
  • $;VERSION$; = 2.0
  • $;RELEASE_TAG$; = $Name$
  • $;REVISION$; = $Revision: 6918 $
  • $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml

[include]




On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:

Hi Gary,

I had a similar problem with that part part of replace_config_keys.pl
(i.e. the add_keys subroutine).  For me anyway, it turned out that the
version of Config::Inifiles that I had gives an error when you try to do
setval() on a value that doesn't already exist.

So I changed the original sub from:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

to:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $cfg->newval($section, $delimeter.$key.$delimeter,$value);
        #$logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

Jason

On 6/7/11 4:11 PM, Gary Xie wrote:
We managed to get the java environment setup and get past the earlier
problem, but now when I create a formatdb pipeline and run it, I get
this error.

more replace_config_keys.stderr
Died with 'Couldn't add key $; PIPELINEID $;=22 to section
[component]' at /opt/ergatis/bin/replace_config_keys.pl line 241


and if I look at the  formatdb.default.user.config
it doesn't contain the key, so it can't replace it.


But, if I look at  ergatis/docs/formatdb.config, it does have the keys
(with no values):

;the following keys are replaced at runtime by the invocation script
$;COMPONENT_CONFIG$; =
$;COMPONENT_XML$; =
$;PIPELINE_XML$; =
$;PIPELINEID$; =


I would guess it uses the formatdb.config to create the config file
for the pipeline, but don't understand why I'm losing the keys and
getting this error.
Can anyone help?
Thanks,
Gary

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

Jason Kwan
Hi Gary,

I'm not if this will be the answer to your problem, but I found in my notes that I did an additional thing.  There is another section of replace_config_keys.pl at ~line 62 that looks like this:

#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

Which I changed to this:
#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
    #$logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

However, I think I did this at the same time as the last suggestion, so I never saw the same error that you have.

Jason

On 6/7/11 4:45 PM, Gary Xie wrote:
Thanks, Jason,
Here are details  on the error message I got.
Gary

formatted source viewer: /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.default.user.config

[interface]

  • classification = file / indexing

[parameters]

  • enter T for protein, F for nucleotide
  • $;PROTEIN$; = F

[input]

  • space-separated list
  • $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna

[output]

  • $;OUTPUT_TOKEN$; = default
  • $;OUTPUT_DIRECTORY$; = $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • call it whatever you like
  • $;DATABASE_NAME$; = nucdb

[component]

  • $;COMPONENT_NAME$; = formatdb
  • $;DESCRIPTION$; = none
  • $;WORKFLOW_REPOSITORY$; = $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • $;PIPELINE_TOKEN$; = unnamed
  • The version,revision,tag here is set by an interpolated CVS tag
  • $;VERSION$; = 2.0
  • $;RELEASE_TAG$; = $Name$
  • $;REVISION$; = $Revision: 6918 $
  • $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml

[include]




On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:

Hi Gary,

I had a similar problem with that part part of replace_config_keys.pl
(i.e. the add_keys subroutine).  For me anyway, it turned out that the
version of Config::Inifiles that I had gives an error when you try to do
setval() on a value that doesn't already exist.

So I changed the original sub from:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

to:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $cfg->newval($section, $delimeter.$key.$delimeter,$value);
        #$logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

Jason

On 6/7/11 4:11 PM, Gary Xie wrote:
We managed to get the java environment setup and get past the earlier
problem, but now when I create a formatdb pipeline and run it, I get
this error.

more replace_config_keys.stderr
Died with 'Couldn't add key $; PIPELINEID $;=22 to section
[component]' at /opt/ergatis/bin/replace_config_keys.pl line 241


and if I look at the  formatdb.default.user.config
it doesn't contain the key, so it can't replace it.


But, if I look at  ergatis/docs/formatdb.config, it does have the keys
(with no values):

;the following keys are replaced at runtime by the invocation script
$;COMPONENT_CONFIG$; =
$;COMPONENT_XML$; =
$;PIPELINE_XML$; =
$;PIPELINEID$; =


I would guess it uses the formatdb.config to create the config file
for the pipeline, but don't understand why I'm losing the keys and
getting this error.
Can anyone help?
Thanks,
Gary

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

gary_xie
Jason, Thanks a lot for your help and insight.
Gary

On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:

Hi Gary,

I'm not if this will be the answer to your problem, but I found in my notes that I did an additional thing.  There is another section of replace_config_keys.pl at ~line 62 that looks like this:

#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

Which I changed to this:
#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
    #$logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

However, I think I did this at the same time as the last suggestion, so I never saw the same error that you have.

Jason

On 6/7/11 4:45 PM, Gary Xie wrote:
Thanks, Jason,
Here are details  on the error message I got.
Gary

formatted source viewer: /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.default.user.config

[interface]

  • classification = file / indexing

[parameters]

  • enter T for protein, F for nucleotide
  • $;PROTEIN$; = F

[input]

  • space-separated list
  • $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna

[output]

  • $;OUTPUT_TOKEN$; = default
  • $;OUTPUT_DIRECTORY$; = $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • call it whatever you like
  • $;DATABASE_NAME$; = nucdb

[component]

  • $;COMPONENT_NAME$; = formatdb
  • $;DESCRIPTION$; = none
  • $;WORKFLOW_REPOSITORY$; = $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • $;PIPELINE_TOKEN$; = unnamed
  • The version,revision,tag here is set by an interpolated CVS tag
  • $;VERSION$; = 2.0
  • $;RELEASE_TAG$; = $Name$
  • $;REVISION$; = $Revision: 6918 $
  • $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml

[include]




On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:

Hi Gary,

I had a similar problem with that part part of replace_config_keys.pl
(i.e. the add_keys subroutine).  For me anyway, it turned out that the
version of Config::Inifiles that I had gives an error when you try to do
setval() on a value that doesn't already exist.

So I changed the original sub from:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

to:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $cfg->newval($section, $delimeter.$key.$delimeter,$value);
        #$logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

Jason

On 6/7/11 4:11 PM, Gary Xie wrote:
We managed to get the java environment setup and get past the earlier
problem, but now when I create a formatdb pipeline and run it, I get
this error.

more replace_config_keys.stderr
Died with 'Couldn't add key $; PIPELINEID $;=22 to section
[component]' at /opt/ergatis/bin/replace_config_keys.pl line 241


and if I look at the  formatdb.default.user.config
it doesn't contain the key, so it can't replace it.


But, if I look at  ergatis/docs/formatdb.config, it does have the keys
(with no values):

;the following keys are replaced at runtime by the invocation script
$;COMPONENT_CONFIG$; =
$;COMPONENT_XML$; =
$;PIPELINE_XML$; =
$;PIPELINEID$; =


I would guess it uses the formatdb.config to create the config file
for the pipeline, but don't understand why I'm losing the keys and
getting this error.
Can anyone help?
Thanks,
Gary

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

Jason Kwan
Hi Everyone,

I wonder what the Devs think of the error that Gary and I encountered.  I suspect maybe Config::Inifiles used to allow setval() on something that didn't previously exist, and if so this should count as a bug?

Jason

On 6/7/11 5:12 PM, Gary Xie wrote:
Jason, Thanks a lot for your help and insight.
Gary

On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:

Hi Gary,

I'm not if this will be the answer to your problem, but I found in my notes that I did an additional thing.  There is another section of replace_config_keys.pl at ~line 62 that looks like this:

#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

Which I changed to this:
#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
    #$logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

However, I think I did this at the same time as the last suggestion, so I never saw the same error that you have.

Jason

On 6/7/11 4:45 PM, Gary Xie wrote:
Thanks, Jason,
Here are details  on the error message I got.
Gary

formatted source viewer: /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.default.user.config

[interface]

  • classification = file / indexing

[parameters]

  • enter T for protein, F for nucleotide
  • $;PROTEIN$; = F

[input]

  • space-separated list
  • $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna

[output]

  • $;OUTPUT_TOKEN$; = default
  • $;OUTPUT_DIRECTORY$; = $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • call it whatever you like
  • $;DATABASE_NAME$; = nucdb

[component]

  • $;COMPONENT_NAME$; = formatdb
  • $;DESCRIPTION$; = none
  • $;WORKFLOW_REPOSITORY$; = $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • $;PIPELINE_TOKEN$; = unnamed
  • The version,revision,tag here is set by an interpolated CVS tag
  • $;VERSION$; = 2.0
  • $;RELEASE_TAG$; = $Name$
  • $;REVISION$; = $Revision: 6918 $
  • $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml

[include]




On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:

Hi Gary,

I had a similar problem with that part part of replace_config_keys.pl
(i.e. the add_keys subroutine).  For me anyway, it turned out that the
version of Config::Inifiles that I had gives an error when you try to do
setval() on a value that doesn't already exist.

So I changed the original sub from:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

to:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $cfg->newval($section, $delimeter.$key.$delimeter,$value);
        #$logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

Jason

On 6/7/11 4:11 PM, Gary Xie wrote:
We managed to get the java environment setup and get past the earlier
problem, but now when I create a formatdb pipeline and run it, I get
this error.

more replace_config_keys.stderr
Died with 'Couldn't add key $; PIPELINEID $;=22 to section
[component]' at /opt/ergatis/bin/replace_config_keys.pl line 241


and if I look at the  formatdb.default.user.config
it doesn't contain the key, so it can't replace it.


But, if I look at  ergatis/docs/formatdb.config, it does have the keys
(with no values):

;the following keys are replaced at runtime by the invocation script
$;COMPONENT_CONFIG$; =
$;COMPONENT_XML$; =
$;PIPELINE_XML$; =
$;PIPELINEID$; =


I would guess it uses the formatdb.config to create the config file
for the pipeline, but don't understand why I'm losing the keys and
getting this error.
Can anyone help?
Thanks,
Gary

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users





------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

Kevin Galens
Yes, this definitely is a bug. Just out of curiosity, what version of the module are you using? It would be great if you setup a ticket on Sourceforge so we can take a look at this issue. Include your proposed changes as well.


Thank you for figuring this out.

Kevin

From: Jason Kwan <[hidden email]>
Organization: University of Utah
Reply-To: <[hidden email]>, <[hidden email]>
Date: Tue, 07 Jun 2011 19:20:55 -0600
To: Gary Xie <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

Hi Everyone,

I wonder what the Devs think of the error that Gary and I encountered.  I suspect maybe Config::Inifiles used to allow setval() on something that didn't previously exist, and if so this should count as a bug?

Jason

On 6/7/11 5:12 PM, Gary Xie wrote:
Jason, Thanks a lot for your help and insight.
Gary

On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:

Hi Gary,

I'm not if this will be the answer to your problem, but I found in my notes that I did an additional thing.  There is another section of replace_config_keys.pl at ~line 62 that looks like this:

#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

Which I changed to this:
#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
    #$logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

However, I think I did this at the same time as the last suggestion, so I never saw the same error that you have.

Jason

On 6/7/11 4:45 PM, Gary Xie wrote:
Thanks, Jason,
Here are details  on the error message I got.
Gary

formatted source viewer: /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.default.user.config

[interface]

  • classification = file / indexing

[parameters]

  • enter T for protein, F for nucleotide
  • $;PROTEIN$; = F

[input]

  • space-separated list
  • $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna

[output]

  • $;OUTPUT_TOKEN$; = default
  • $;OUTPUT_DIRECTORY$; = $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • call it whatever you like
  • $;DATABASE_NAME$; = nucdb

[component]

  • $;COMPONENT_NAME$; = formatdb
  • $;DESCRIPTION$; = none
  • $;WORKFLOW_REPOSITORY$; = $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • $;PIPELINE_TOKEN$; = unnamed
  • The version,revision,tag here is set by an interpolated CVS tag
  • $;VERSION$; = 2.0
  • $;RELEASE_TAG$; = $Name$
  • $;REVISION$; = $Revision: 6918 $
  • $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml

[include]




On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:

Hi Gary,

I had a similar problem with that part part of replace_config_keys.pl
(i.e. the add_keys subroutine).  For me anyway, it turned out that the
version of Config::Inifiles that I had gives an error when you try to do
setval() on a value that doesn't already exist.

So I changed the original sub from:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

to:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $cfg->newval($section, $delimeter.$key.$delimeter,$value);
        #$logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

Jason

On 6/7/11 4:11 PM, Gary Xie wrote:
We managed to get the java environment setup and get past the earlier
problem, but now when I create a formatdb pipeline and run it, I get
this error.

more replace_config_keys.stderr
Died with 'Couldn't add key $; PIPELINEID $;=22 to section
[component]' at /opt/ergatis/bin/replace_config_keys.pl line 241


and if I look at the  formatdb.default.user.config
it doesn't contain the key, so it can't replace it.


But, if I look at  ergatis/docs/formatdb.config, it does have the keys
(with no values):

;the following keys are replaced at runtime by the invocation script
$;COMPONENT_CONFIG$; =
$;COMPONENT_XML$; =
$;PIPELINE_XML$; =
$;PIPELINEID$; =


I would guess it uses the formatdb.config to create the config file
for the pipeline, but don't understand why I'm losing the keys and
getting this error.
Can anyone help?
Thanks,
Gary

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------ EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev_______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

gary_xie
Hi, all,
We hit another glitch; It seems the code that runs the workflow is broken. I was hoping to wait for the new release.  But we could post the issue here:

It looks like whatever is creating the name of the file passed into the script that fails 
 
opt/ergatis/bin/create_file_iterator_list.pl line 140.
 
is not substituting the pipleine id into the file name:
 
the command is:
/opt/ergatis/bin/create_file_iterator_list --input_directory= --input_directory_extension=fsa --input_file=/opt/ergatis/samples/proteins.faa --input_file_list= --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/_default/i1.list --timestamp=0
 
and should be 
/opt/ergatis/bin/create_file_iterator_list --input_directory= --input_directory_extension=fsa --input_file=/opt/ergatis/samples/proteins.faa --input_file_list= --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/10_default/i1.list --timestamp=0
 
I don't know where in the ergatis code this substitution should happen. Any insight?
Thanks,
Gary
 
On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:

Yes, this definitely is a bug. Just out of curiosity, what version of the module are you using? It would be great if you setup a ticket on Sourceforge so we can take a look at this issue. Include your proposed changes as well.


Thank you for figuring this out.

Kevin

From: Jason Kwan <[hidden email]>
Organization: University of Utah
Reply-To: <[hidden email]>, <[hidden email]>
Date: Tue, 07 Jun 2011 19:20:55 -0600
To: Gary Xie <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

Hi Everyone,

I wonder what the Devs think of the error that Gary and I encountered.  I suspect maybe Config::Inifiles used to allow setval() on something that didn't previously exist, and if so this should count as a bug?

Jason

On 6/7/11 5:12 PM, Gary Xie wrote:
Jason, Thanks a lot for your help and insight.
Gary

On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:

Hi Gary,

I'm not if this will be the answer to your problem, but I found in my notes that I did an additional thing.  There is another section of replace_config_keys.pl at ~line 62 that looks like this:

#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

Which I changed to this:
#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
    #$logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

However, I think I did this at the same time as the last suggestion, so I never saw the same error that you have.

Jason

On 6/7/11 4:45 PM, Gary Xie wrote:
Thanks, Jason,
Here are details  on the error message I got.
Gary

formatted source viewer: /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.default.user.config

[interface]

  • classification = file / indexing

[parameters]

  • enter T for protein, F for nucleotide
  • $;PROTEIN$; = F

[input]

  • space-separated list
  • $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna

[output]

  • $;OUTPUT_TOKEN$; = default
  • $;OUTPUT_DIRECTORY$; = $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • call it whatever you like
  • $;DATABASE_NAME$; = nucdb

[component]

  • $;COMPONENT_NAME$; = formatdb
  • $;DESCRIPTION$; = none
  • $;WORKFLOW_REPOSITORY$; = $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • $;PIPELINE_TOKEN$; = unnamed
  • The version,revision,tag here is set by an interpolated CVS tag
  • $;VERSION$; = 2.0
  • $;RELEASE_TAG$; = $Name$
  • $;REVISION$; = $Revision: 6918 $
  • $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml

[include]




On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:

Hi Gary,

I had a similar problem with that part part of replace_config_keys.pl
(i.e. the add_keys subroutine).  For me anyway, it turned out that the
version of Config::Inifiles that I had gives an error when you try to do
setval() on a value that doesn't already exist.

So I changed the original sub from:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

to:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $cfg->newval($section, $delimeter.$key.$delimeter,$value);
        #$logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

Jason

On 6/7/11 4:11 PM, Gary Xie wrote:
We managed to get the java environment setup and get past the earlier
problem, but now when I create a formatdb pipeline and run it, I get
this error.

more replace_config_keys.stderr
Died with 'Couldn't add key $; PIPELINEID $;=22 to section
[component]' at /opt/ergatis/bin/replace_config_keys.pl line 241


and if I look at the  formatdb.default.user.config
it doesn't contain the key, so it can't replace it.


But, if I look at  ergatis/docs/formatdb.config, it does have the keys
(with no values):

;the following keys are replaced at runtime by the invocation script
$;COMPONENT_CONFIG$; =
$;COMPONENT_XML$; =
$;PIPELINE_XML$; =
$;PIPELINEID$; =


I would guess it uses the formatdb.config to create the config file
for the pipeline, but don't understand why I'm losing the keys and
getting this error.
Can anyone help?
Thanks,
Gary

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------ EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev_______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

kmuthuraman

Hello All

Thanks for your responses.

I have narrowed down the possible causes for the errors.

 

1.       When I run the shell script under workflows/scripts, it does not produce any output. But when I run the final command “RunWorkflow” in the script, it produces the log file. Meaning the environment variables created in the script are wrong. However when I open a new shell and try running the command again, it does not work. So, there is some environment variable that is working in the current shell I’m working on, and I don’t know what that is. The only environment variable I added to the apache conf was PERL%LIB.

2.       Even when I ran the RunWorkflow command manually, I got this error :

[root@nextgen ergatis]# cat projects/maize/workflow/runtime/formatdb/15_default/replace_config_keys.stderr

Died with 'Couldn't add key $; PIPELINEID $;=15 to section [component]' at ../ergatis/bin/replace_config_keys.pl line 241

 

I’m just trying to run formatdb on a fasta file.

 

 

Sincerely,
Karthik Narayan

 

From: Gary Xie [mailto:[hidden email]]
Sent: Tuesday, June 14, 2011 11:04 AM
To: Kevin Galens
Cc: [hidden email]
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi, all,

We hit another glitch; It seems the code that runs the workflow is broken. I was hoping to wait for the new release.  But we could post the issue here:

 

It looks like whatever is creating the name of the file passed into the script that fails 

 

opt/ergatis/bin/create_file_iterator_list.pl line 140.

 

is not substituting the pipleine id into the file name:

 

the command is:

/opt/ergatis/bin/create_file_iterator_list --input_directory= --input_directory_extension=fsa --input_file=/opt/ergatis/samples/proteins.faa --input_file_list= --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/_default/i1.list --timestamp=0

 

and should be 

/opt/ergatis/bin/create_file_iterator_list --input_directory= --input_directory_extension=fsa --input_file=/opt/ergatis/samples/proteins.faa --input_file_list= --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/10_default/i1.list --timestamp=0

 

I don't know where in the ergatis code this substitution should happen. Any insight?

Thanks,

Gary

 

On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:



Yes, this definitely is a bug. Just out of curiosity, what version of the module are you using? It would be great if you setup a ticket on Sourceforge so we can take a look at this issue. Include your proposed changes as well.

 

 

Thank you for figuring this out.

 

Kevin

 

From: Jason Kwan <[hidden email]>
Organization: University of Utah
Reply-To: <[hidden email]>, <[hidden email]>
Date: Tue, 07 Jun 2011 19:20:55 -0600
To: Gary Xie <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi Everyone,

I wonder what the Devs think of the error that Gary and I encountered.  I suspect maybe Config::Inifiles used to allow setval() on something that didn't previously exist, and if so this should count as a bug?

Jason

On 6/7/11 5:12 PM, Gary Xie wrote:

Jason, Thanks a lot for your help and insight.

Gary

 

On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:



Hi Gary,

I'm not if this will be the answer to your problem, but I found in my notes that I did an additional thing.  There is another section of replace_config_keys.pl at ~line 62 that looks like this:

#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

Which I changed to this:
#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
    #$logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

However, I think I did this at the same time as the last suggestion, so I never saw the same error that you have.

Jason

On 6/7/11 4:45 PM, Gary Xie wrote:

Thanks, Jason,

Here are details  on the error message I got.

Gary

 

formatted source viewer: /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.default.user.config

[interface]

  • classification = file / indexing

[parameters]

  • enter T for protein, F for nucleotide
  • $;PROTEIN$; = F

[input]

  • space-separated list
  • $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna

[output]

  • $;OUTPUT_TOKEN$; = default
  • $;OUTPUT_DIRECTORY$; = $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • call it whatever you like
  • $;DATABASE_NAME$; = nucdb

[component]

  • $;COMPONENT_NAME$; = formatdb
  • $;DESCRIPTION$; = none
  • $;WORKFLOW_REPOSITORY$; = $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • $;PIPELINE_TOKEN$; = unnamed
  • The version,revision,tag here is set by an interpolated CVS tag
  • $;VERSION$; = 2.0
  • $;RELEASE_TAG$; = $Name$
  • $;REVISION$; = $Revision: 6918 $
  • $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml

[include]

 

 

 

On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:



Hi Gary,

I had a similar problem with that part part of replace_config_keys.pl
(i.e. the add_keys subroutine).  For me anyway, it turned out that the
version of Config::Inifiles that I had gives an error when you try to do
setval() on a value that doesn't already exist.

So I changed the original sub from:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

to:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $cfg->newval($section, $delimeter.$key.$delimeter,$value);
        #$logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

Jason

On 6/7/11 4:11 PM, Gary Xie wrote:

We managed to get the java environment setup and get past the earlier

problem, but now when I create a formatdb pipeline and run it, I get

this error.

 

more replace_config_keys.stderr

Died with 'Couldn't add key $; PIPELINEID $;=22 to section

[component]' at /opt/ergatis/bin/replace_config_keys.pl line 241

 

 

and if I look at the  formatdb.default.user.config

it doesn't contain the key, so it can't replace it.

 

 

But, if I look at  ergatis/docs/formatdb.config, it does have the keys

(with no values):

 

;the following keys are replaced at runtime by the invocation script

$;COMPONENT_CONFIG$; =

$;COMPONENT_XML$; =

$;PIPELINE_XML$; =

$;PIPELINEID$; =

 

 

I would guess it uses the formatdb.config to create the config file

for the pipeline, but don't understand why I'm losing the keys and

getting this error.

Can anyone help?

Thanks,

Gary

 

------------------------------------------------------------------------------

EditLive Enterprise is the world's most technically advanced content

authoring tool. Experience the power of Track Changes, Inline Image

Editing and ensure content is compliant with Accessibility Checking.

http://p.sf.net/sfu/ephox-dev2dev

_______________________________________________

Ergatis-users mailing list

[hidden email]

https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

 

 

 

 

------------------------------------------------------------------------------ EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev_______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users

 


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

Jason Kwan
Hi Karthik,

This could be the bug that was described on this list on the 7th and 8th of June, please look back in the archives and let us know if it fixes it.

Thanks

Jason


On 6/14/11 9:50 AM, Muthuraman, Karthik Narayan (K) wrote:

Hello All

Thanks for your responses.

I have narrowed down the possible causes for the errors.

 

1.       When I run the shell script under workflows/scripts, it does not produce any output. But when I run the final command “RunWorkflow” in the script, it produces the log file. Meaning the environment variables created in the script are wrong. However when I open a new shell and try running the command again, it does not work. So, there is some environment variable that is working in the current shell I’m working on, and I don’t know what that is. The only environment variable I added to the apache conf was PERL%LIB.

2.       Even when I ran the RunWorkflow command manually, I got this error :

[root@nextgen ergatis]# cat projects/maize/workflow/runtime/formatdb/15_default/replace_config_keys.stderr

Died with 'Couldn't add key $; PIPELINEID $;=15 to section [component]' at ../ergatis/bin/replace_config_keys.pl line 241

 

I’m just trying to run formatdb on a fasta file.

 

 

Sincerely,
Karthik Narayan

 

From: Gary Xie [[hidden email]]
Sent: Tuesday, June 14, 2011 11:04 AM
To: Kevin Galens
Cc: [hidden email]
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi, all,

We hit another glitch; It seems the code that runs the workflow is broken. I was hoping to wait for the new release.  But we could post the issue here:

 

It looks like whatever is creating the name of the file passed into the script that fails 

 

opt/ergatis/bin/create_file_iterator_list.pl line 140.

 

is not substituting the pipleine id into the file name:

 

the command is:

/opt/ergatis/bin/create_file_iterator_list --input_directory= --input_directory_extension=fsa --input_file=/opt/ergatis/samples/proteins.faa --input_file_list= --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/_default/i1.list --timestamp=0

 

and should be 

/opt/ergatis/bin/create_file_iterator_list --input_directory= --input_directory_extension=fsa --input_file=/opt/ergatis/samples/proteins.faa --input_file_list= --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/10_default/i1.list --timestamp=0

 

I don't know where in the ergatis code this substitution should happen. Any insight?

Thanks,

Gary

 

On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:



Yes, this definitely is a bug. Just out of curiosity, what version of the module are you using? It would be great if you setup a ticket on Sourceforge so we can take a look at this issue. Include your proposed changes as well.

 

 

Thank you for figuring this out.

 

Kevin

 

From: Jason Kwan <[hidden email]>
Organization: University of Utah
Reply-To: <[hidden email]>, <[hidden email]>
Date: Tue, 07 Jun 2011 19:20:55 -0600
To: Gary Xie <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi Everyone,

I wonder what the Devs think of the error that Gary and I encountered.  I suspect maybe Config::Inifiles used to allow setval() on something that didn't previously exist, and if so this should count as a bug?

Jason

On 6/7/11 5:12 PM, Gary Xie wrote:

Jason, Thanks a lot for your help and insight.

Gary

 

On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:



Hi Gary,

I'm not if this will be the answer to your problem, but I found in my notes that I did an additional thing.  There is another section of replace_config_keys.pl at ~line 62 that looks like this:

#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

Which I changed to this:
#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
    #$logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

However, I think I did this at the same time as the last suggestion, so I never saw the same error that you have.

Jason

On 6/7/11 4:45 PM, Gary Xie wrote:

Thanks, Jason,

Here are details  on the error message I got.

Gary

 

formatted source viewer: /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.default.user.config

[interface]

  • classification = file / indexing

[parameters]

  • enter T for protein, F for nucleotide
  • $;PROTEIN$; = F

[input]

  • space-separated list
  • $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna

[output]

  • $;OUTPUT_TOKEN$; = default
  • $;OUTPUT_DIRECTORY$; = $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • call it whatever you like
  • $;DATABASE_NAME$; = nucdb

[component]

  • $;COMPONENT_NAME$; = formatdb
  • $;DESCRIPTION$; = none
  • $;WORKFLOW_REPOSITORY$; = $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • $;PIPELINE_TOKEN$; = unnamed
  • The version,revision,tag here is set by an interpolated CVS tag
  • $;VERSION$; = 2.0
  • $;RELEASE_TAG$; = $Name$
  • $;REVISION$; = $Revision: 6918 $
  • $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml

[include]

 

 

 

On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:



Hi Gary,

I had a similar problem with that part part of replace_config_keys.pl
(i.e. the add_keys subroutine).  For me anyway, it turned out that the
version of Config::Inifiles that I had gives an error when you try to do
setval() on a value that doesn't already exist.

So I changed the original sub from:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

to:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $cfg->newval($section, $delimeter.$key.$delimeter,$value);
        #$logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

Jason

On 6/7/11 4:11 PM, Gary Xie wrote:

We managed to get the java environment setup and get past the earlier

problem, but now when I create a formatdb pipeline and run it, I get

this error.

 

more replace_config_keys.stderr

Died with 'Couldn't add key $; PIPELINEID $;=22 to section

[component]' at /opt/ergatis/bin/replace_config_keys.pl line 241

 

 

and if I look at the  formatdb.default.user.config

it doesn't contain the key, so it can't replace it.

 

 

But, if I look at  ergatis/docs/formatdb.config, it does have the keys

(with no values):

 

;the following keys are replaced at runtime by the invocation script

$;COMPONENT_CONFIG$; =

$;COMPONENT_XML$; =

$;PIPELINE_XML$; =

$;PIPELINEID$; =

 

 

I would guess it uses the formatdb.config to create the config file

for the pipeline, but don't understand why I'm losing the keys and

getting this error.

Can anyone help?

Thanks,

Gary

 

------------------------------------------------------------------------------

EditLive Enterprise is the world's most technically advanced content

authoring tool. Experience the power of Track Changes, Inline Image

Editing and ensure content is compliant with Accessibility Checking.

http://p.sf.net/sfu/ephox-dev2dev

_______________________________________________

Ergatis-users mailing list

[hidden email]

https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

 

 

 

 

------------------------------------------------------------------------------ EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev_______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users

 



------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

kmuthuraman

Hello Jason

I looked at the thread, and patches proposed did not fix my problem.

 

There are 2 params in the repace_config_keys.pl, that do not exist. They are:

COMPONENT_XML=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_default/component.xml -- output_conf=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_default/formatdb.default.final.config

 

These two files are not created. Any thoughts ?

 

Sincerely,
Karthik Narayan

 

From: Jason Kwan [mailto:[hidden email]]
Sent: Tuesday, June 14, 2011 1:15 PM
To: [hidden email]
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi Karthik,

This could be the bug that was described on this list on the 7th and 8th of June, please look back in the archives and let us know if it fixes it.

Thanks

Jason


On 6/14/11 9:50 AM, Muthuraman, Karthik Narayan (K) wrote:

Hello All

Thanks for your responses.

I have narrowed down the possible causes for the errors.

 

When I run the shell script under workflows/scripts, it does not produce any output. But when I run the final command “RunWorkflow” in the script, it produces the log file. Meaning the environment variables created in the script are wrong. However when I open a new shell and try running the command again, it does not work. So, there is some environment variable that is working in the current shell I’m working on, and I don’t know what that is. The only environment variable I added to the apache conf was PERL%LIB.

Even when I ran the RunWorkflow command manually, I got this error :

[root@nextgen ergatis]# cat projects/maize/workflow/runtime/formatdb/15_default/replace_config_keys.stderr

Died with 'Couldn't add key $; PIPELINEID $;=15 to section [component]' at ../ergatis/bin/replace_config_keys.pl line 241

 

I’m just trying to run formatdb on a fasta file.

 

 

Sincerely,
Karthik Narayan

 

From: Gary Xie [[hidden email]]
Sent: Tuesday, June 14, 2011 11:04 AM
To: Kevin Galens
Cc: [hidden email]
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi, all,

We hit another glitch; It seems the code that runs the workflow is broken. I was hoping to wait for the new release.  But we could post the issue here:

 

It looks like whatever is creating the name of the file passed into the script that fails 

 

opt/ergatis/bin/create_file_iterator_list.pl line 140.

 

is not substituting the pipleine id into the file name:

 

the command is:

/opt/ergatis/bin/create_file_iterator_list --input_directory= --input_directory_extension=fsa --input_file=/opt/ergatis/samples/proteins.faa --input_file_list= --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/_default/i1.list --timestamp=0

 

and should be 

/opt/ergatis/bin/create_file_iterator_list --input_directory= --input_directory_extension=fsa --input_file=/opt/ergatis/samples/proteins.faa --input_file_list= --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/10_default/i1.list --timestamp=0

 

I don't know where in the ergatis code this substitution should happen. Any insight?

Thanks,

Gary

 

On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:




Yes, this definitely is a bug. Just out of curiosity, what version of the module are you using? It would be great if you setup a ticket on Sourceforge so we can take a look at this issue. Include your proposed changes as well.

 

 

Thank you for figuring this out.

 

Kevin

 

From: Jason Kwan <[hidden email]>
Organization: University of Utah
Reply-To: <[hidden email]>, <[hidden email]>
Date: Tue, 07 Jun 2011 19:20:55 -0600
To: Gary Xie <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi Everyone,

I wonder what the Devs think of the error that Gary and I encountered.  I suspect maybe Config::Inifiles used to allow setval() on something that didn't previously exist, and if so this should count as a bug?

Jason

On 6/7/11 5:12 PM, Gary Xie wrote:

Jason, Thanks a lot for your help and insight.

Gary

 

On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:




Hi Gary,

I'm not if this will be the answer to your problem, but I found in my notes that I did an additional thing.  There is another section of replace_config_keys.pl at ~line 62 that looks like this:

#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

Which I changed to this:
#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
    #$logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

However, I think I did this at the same time as the last suggestion, so I never saw the same error that you have.

Jason

On 6/7/11 4:45 PM, Gary Xie wrote:

Thanks, Jason,

Here are details  on the error message I got.

Gary

 

formatted source viewer: /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.default.user.config

[interface]

  • classification = file / indexing

[parameters]

  • enter T for protein, F for nucleotide
  • $;PROTEIN$; = F

[input]

  • space-separated list
  • $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna

[output]

  • $;OUTPUT_TOKEN$; = default
  • $;OUTPUT_DIRECTORY$; = $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • call it whatever you like
  • $;DATABASE_NAME$; = nucdb

[component]

  • $;COMPONENT_NAME$; = formatdb
  • $;DESCRIPTION$; = none
  • $;WORKFLOW_REPOSITORY$; = $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • $;PIPELINE_TOKEN$; = unnamed
  • The version,revision,tag here is set by an interpolated CVS tag
  • $;VERSION$; = 2.0
  • $;RELEASE_TAG$; = $Name$
  • $;REVISION$; = $Revision: 6918 $
  • $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml

[include]

 

 

 

On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:




Hi Gary,

I had a similar problem with that part part of replace_config_keys.pl
(i.e. the add_keys subroutine).  For me anyway, it turned out that the
version of Config::Inifiles that I had gives an error when you try to do
setval() on a value that doesn't already exist.

So I changed the original sub from:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

to:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $cfg->newval($section, $delimeter.$key.$delimeter,$value);
        #$logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

Jason

On 6/7/11 4:11 PM, Gary Xie wrote:


We managed to get the java environment setup and get past the earlier

problem, but now when I create a formatdb pipeline and run it, I get

this error.

 

more replace_config_keys.stderr

Died with 'Couldn't add key $; PIPELINEID $;=22 to section

[component]' at /opt/ergatis/bin/replace_config_keys.pl line 241

 

 

and if I look at the  formatdb.default.user.config

it doesn't contain the key, so it can't replace it.

 

 

But, if I look at  ergatis/docs/formatdb.config, it does have the keys

(with no values):

 

;the following keys are replaced at runtime by the invocation script

$;COMPONENT_CONFIG$; =

$;COMPONENT_XML$; =

$;PIPELINE_XML$; =

$;PIPELINEID$; =

 

 

I would guess it uses the formatdb.config to create the config file

for the pipeline, but don't understand why I'm losing the keys and

getting this error.

Can anyone help?

Thanks,

Gary

 

------------------------------------------------------------------------------

EditLive Enterprise is the world's most technically advanced content

authoring tool. Experience the power of Track Changes, Inline Image

Editing and ensure content is compliant with Accessibility Checking.

http://p.sf.net/sfu/ephox-dev2dev

_______________________________________________

Ergatis-users mailing list

[hidden email]

https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

 

 

 

 

------------------------------------------------------------------------------ EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev_______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users

 

 


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

Jason Kwan
Hi Karthik,

You got past the "create output directory" and "create temp directory" stages so I can conclude that you probably have the right permissions for the output directory (do these get created)?  Other than that there is not much to go on, apart from exactly the same error message that Gary faced.  If I was faced with this situation, I would double check that I had changed replace_config_keys.pl in the install directory, rather than the directory that I had unpacked the ergatis source package (sudo find / -name "replace_config_keys.pl"), then change all of them just to be safe (and keep a backup of the original of course). 

Other than that, I can't think of anything.  Perhaps the devs have some further thoughts?

Jason

On 6/14/11 11:42 AM, Muthuraman, Karthik Narayan (K) wrote:

Hello Jason

I looked at the thread, and patches proposed did not fix my problem.

 

There are 2 params in the repace_config_keys.pl, that do not exist. They are:

COMPONENT_XML=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_default/component.xml -- output_conf=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_default/formatdb.default.final.config

 

These two files are not created. Any thoughts ?

 

Sincerely,
Karthik Narayan

 

From: Jason Kwan [[hidden email]]
Sent: Tuesday, June 14, 2011 1:15 PM
To: [hidden email]
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi Karthik,

This could be the bug that was described on this list on the 7th and 8th of June, please look back in the archives and let us know if it fixes it.

Thanks

Jason


On 6/14/11 9:50 AM, Muthuraman, Karthik Narayan (K) wrote:

Hello All

Thanks for your responses.

I have narrowed down the possible causes for the errors.

 

When I run the shell script under workflows/scripts, it does not produce any output. But when I run the final command “RunWorkflow” in the script, it produces the log file. Meaning the environment variables created in the script are wrong. However when I open a new shell and try running the command again, it does not work. So, there is some environment variable that is working in the current shell I’m working on, and I don’t know what that is. The only environment variable I added to the apache conf was PERL%LIB.

Even when I ran the RunWorkflow command manually, I got this error :

[root@nextgen ergatis]# cat projects/maize/workflow/runtime/formatdb/15_default/replace_config_keys.stderr

Died with 'Couldn't add key $; PIPELINEID $;=15 to section [component]' at ../ergatis/bin/replace_config_keys.pl line 241

 

I’m just trying to run formatdb on a fasta file.

 

 

Sincerely,
Karthik Narayan

 

From: Gary Xie [[hidden email]]
Sent: Tuesday, June 14, 2011 11:04 AM
To: Kevin Galens
Cc: [hidden email]
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi, all,

We hit another glitch; It seems the code that runs the workflow is broken. I was hoping to wait for the new release.  But we could post the issue here:

 

It looks like whatever is creating the name of the file passed into the script that fails 

 

opt/ergatis/bin/create_file_iterator_list.pl line 140.

 

is not substituting the pipleine id into the file name:

 

the command is:

/opt/ergatis/bin/create_file_iterator_list --input_directory= --input_directory_extension=fsa --input_file=/opt/ergatis/samples/proteins.faa --input_file_list= --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/_default/i1.list --timestamp=0

 

and should be 

/opt/ergatis/bin/create_file_iterator_list --input_directory= --input_directory_extension=fsa --input_file=/opt/ergatis/samples/proteins.faa --input_file_list= --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/10_default/i1.list --timestamp=0

 

I don't know where in the ergatis code this substitution should happen. Any insight?

Thanks,

Gary

 

On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:




Yes, this definitely is a bug. Just out of curiosity, what version of the module are you using? It would be great if you setup a ticket on Sourceforge so we can take a look at this issue. Include your proposed changes as well.

 

 

Thank you for figuring this out.

 

Kevin

 

From: Jason Kwan <[hidden email]>
Organization: University of Utah
Reply-To: <[hidden email]>, <[hidden email]>
Date: Tue, 07 Jun 2011 19:20:55 -0600
To: Gary Xie <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi Everyone,

I wonder what the Devs think of the error that Gary and I encountered.  I suspect maybe Config::Inifiles used to allow setval() on something that didn't previously exist, and if so this should count as a bug?

Jason

On 6/7/11 5:12 PM, Gary Xie wrote:

Jason, Thanks a lot for your help and insight.

Gary

 

On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:




Hi Gary,

I'm not if this will be the answer to your problem, but I found in my notes that I did an additional thing.  There is another section of replace_config_keys.pl at ~line 62 that looks like this:

#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

Which I changed to this:
#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
    #$logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

However, I think I did this at the same time as the last suggestion, so I never saw the same error that you have.

Jason

On 6/7/11 4:45 PM, Gary Xie wrote:

Thanks, Jason,

Here are details  on the error message I got.

Gary

 

formatted source viewer: /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.default.user.config

[interface]

  • classification = file / indexing

[parameters]

  • enter T for protein, F for nucleotide
  • $;PROTEIN$; = F

[input]

  • space-separated list
  • $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna

[output]

  • $;OUTPUT_TOKEN$; = default
  • $;OUTPUT_DIRECTORY$; = $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • call it whatever you like
  • $;DATABASE_NAME$; = nucdb

[component]

  • $;COMPONENT_NAME$; = formatdb
  • $;DESCRIPTION$; = none
  • $;WORKFLOW_REPOSITORY$; = $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • $;PIPELINE_TOKEN$; = unnamed
  • The version,revision,tag here is set by an interpolated CVS tag
  • $;VERSION$; = 2.0
  • $;RELEASE_TAG$; = $Name$
  • $;REVISION$; = $Revision: 6918 $
  • $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml

[include]

 

 

 

On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:




Hi Gary,

I had a similar problem with that part part of replace_config_keys.pl
(i.e. the add_keys subroutine).  For me anyway, it turned out that the
version of Config::Inifiles that I had gives an error when you try to do
setval() on a value that doesn't already exist.

So I changed the original sub from:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

to:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $cfg->newval($section, $delimeter.$key.$delimeter,$value);
        #$logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

Jason

On 6/7/11 4:11 PM, Gary Xie wrote:


We managed to get the java environment setup and get past the earlier

problem, but now when I create a formatdb pipeline and run it, I get

this error.

 

more replace_config_keys.stderr

Died with 'Couldn't add key $; PIPELINEID $;=22 to section

[component]' at /opt/ergatis/bin/replace_config_keys.pl line 241

 

 

and if I look at the  formatdb.default.user.config

it doesn't contain the key, so it can't replace it.

 

 

But, if I look at  ergatis/docs/formatdb.config, it does have the keys

(with no values):

 

;the following keys are replaced at runtime by the invocation script

$;COMPONENT_CONFIG$; =

$;COMPONENT_XML$; =

$;PIPELINE_XML$; =

$;PIPELINEID$; =

 

 

I would guess it uses the formatdb.config to create the config file

for the pipeline, but don't understand why I'm losing the keys and

getting this error.

Can anyone help?

Thanks,

Gary

 

------------------------------------------------------------------------------

EditLive Enterprise is the world's most technically advanced content

authoring tool. Experience the power of Track Changes, Inline Image

Editing and ensure content is compliant with Accessibility Checking.

http://p.sf.net/sfu/ephox-dev2dev

_______________________________________________

Ergatis-users mailing list

[hidden email]

https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

 

 

 

 

------------------------------------------------------------------------------ EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev_______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users

 

 



------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

kmuthuraman

Thanks Jason. Also, when I run the shell script “pipeline.32.run.sh” under workflows/scripts from my shell, it createdsthe log file, though with errors.

But when ergatis runs it as apache, it does not create the log file at all.

I have provided all the environment vars.

 

Sincerely,
Karthik Narayan

 

From: Jason Kwan [mailto:[hidden email]]
Sent: Tuesday, June 14, 2011 2:13 PM
To: Muthuraman, Karthik Narayan (K)
Cc: [hidden email]
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi Karthik,

You got past the "create output directory" and "create temp directory" stages so I can conclude that you probably have the right permissions for the output directory (do these get created)?  Other than that there is not much to go on, apart from exactly the same error message that Gary faced.  If I was faced with this situation, I would double check that I had changed replace_config_keys.pl in the install directory, rather than the directory that I had unpacked the ergatis source package (sudo find / -name "replace_config_keys.pl"), then change all of them just to be safe (and keep a backup of the original of course). 

Other than that, I can't think of anything.  Perhaps the devs have some further thoughts?

Jason

On 6/14/11 11:42 AM, Muthuraman, Karthik Narayan (K) wrote:

Hello Jason

I looked at the thread, and patches proposed did not fix my problem.

 

There are 2 params in the repace_config_keys.pl, that do not exist. They are:

COMPONENT_XML=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_default/component.xml -- output_conf=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_default/formatdb.default.final.config

 

These two files are not created. Any thoughts ?

 

Sincerely,
Karthik Narayan

 

From: Jason Kwan [[hidden email]]
Sent: Tuesday, June 14, 2011 1:15 PM
To: [hidden email]
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi Karthik,

This could be the bug that was described on this list on the 7th and 8th of June, please look back in the archives and let us know if it fixes it.

Thanks

Jason


On 6/14/11 9:50 AM, Muthuraman, Karthik Narayan (K) wrote:

Hello All

Thanks for your responses.

I have narrowed down the possible causes for the errors.

 

When I run the shell script under workflows/scripts, it does not produce any output. But when I run the final command “RunWorkflow” in the script, it produces the log file. Meaning the environment variables created in the script are wrong. However when I open a new shell and try running the command again, it does not work. So, there is some environment variable that is working in the current shell I’m working on, and I don’t know what that is. The only environment variable I added to the apache conf was PERL%LIB.

Even when I ran the RunWorkflow command manually, I got this error :

[root@nextgen ergatis]# cat projects/maize/workflow/runtime/formatdb/15_default/replace_config_keys.stderr

Died with 'Couldn't add key $; PIPELINEID $;=15 to section [component]' at ../ergatis/bin/replace_config_keys.pl line 241

 

I’m just trying to run formatdb on a fasta file.

 

 

Sincerely,
Karthik Narayan

 

From: Gary Xie [[hidden email]]
Sent: Tuesday, June 14, 2011 11:04 AM
To: Kevin Galens
Cc: [hidden email]
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi, all,

We hit another glitch; It seems the code that runs the workflow is broken. I was hoping to wait for the new release.  But we could post the issue here:

 

It looks like whatever is creating the name of the file passed into the script that fails 

 

opt/ergatis/bin/create_file_iterator_list.pl line 140.

 

is not substituting the pipleine id into the file name:

 

the command is:

/opt/ergatis/bin/create_file_iterator_list --input_directory= --input_directory_extension=fsa --input_file=/opt/ergatis/samples/proteins.faa --input_file_list= --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/_default/i1.list --timestamp=0

 

and should be 

/opt/ergatis/bin/create_file_iterator_list --input_directory= --input_directory_extension=fsa --input_file=/opt/ergatis/samples/proteins.faa --input_file_list= --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/10_default/i1.list --timestamp=0

 

I don't know where in the ergatis code this substitution should happen. Any insight?

Thanks,

Gary

 

On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:





Yes, this definitely is a bug. Just out of curiosity, what version of the module are you using? It would be great if you setup a ticket on Sourceforge so we can take a look at this issue. Include your proposed changes as well.

 

 

Thank you for figuring this out.

 

Kevin

 

From: Jason Kwan <[hidden email]>
Organization: University of Utah
Reply-To: <[hidden email]>, <[hidden email]>
Date: Tue, 07 Jun 2011 19:20:55 -0600
To: Gary Xie <[hidden email]>
Cc: "[hidden email]" <[hidden email]>
Subject: Re: [Ergatis-users] error message on the one-step workflow: formatdb

 

Hi Everyone,

I wonder what the Devs think of the error that Gary and I encountered.  I suspect maybe Config::Inifiles used to allow setval() on something that didn't previously exist, and if so this should count as a bug?

Jason

On 6/7/11 5:12 PM, Gary Xie wrote:

Jason, Thanks a lot for your help and insight.

Gary

 

On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:





Hi Gary,

I'm not if this will be the answer to your problem, but I found in my notes that I did an additional thing.  There is another section of replace_config_keys.pl at ~line 62 that looks like this:

#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

Which I changed to this:
#Write the output location of this file as key $;COMPONENT_CONFIG$;
my $ret = $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
if(!$ret){
    $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
    #$logger->logdie("Couldn't add key ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to section [component]");
}
&check_parameters($cfg,'component');

However, I think I did this at the same time as the last suggestion, so I never saw the same error that you have.

Jason

On 6/7/11 4:45 PM, Gary Xie wrote:

Thanks, Jason,

Here are details  on the error message I got.

Gary

 

formatted source viewer: /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.default.user.config

[interface]

  • classification = file / indexing

[parameters]

  • enter T for protein, F for nucleotide
  • $;PROTEIN$; = F

[input]

  • space-separated list
  • $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna

[output]

  • $;OUTPUT_TOKEN$; = default
  • $;OUTPUT_DIRECTORY$; = $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • call it whatever you like
  • $;DATABASE_NAME$; = nucdb

[component]

  • $;COMPONENT_NAME$; = formatdb
  • $;DESCRIPTION$; = none
  • $;WORKFLOW_REPOSITORY$; = $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
  • $;PIPELINE_TOKEN$; = unnamed
  • The version,revision,tag here is set by an interpolated CVS tag
  • $;VERSION$; = 2.0
  • $;RELEASE_TAG$; = $Name$
  • $;REVISION$; = $Revision: 6918 $
  • $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml

[include]

 

 

 

On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:





Hi Gary,

I had a similar problem with that part part of replace_config_keys.pl
(i.e. the add_keys subroutine).  For me anyway, it turned out that the
version of Config::Inifiles that I had gives an error when you try to do
setval() on a value that doesn't already exist.

So I changed the original sub from:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

to:
sub add_keys{
    my($cfg,$section,@keys) = @_;
    $logger->debug("Adding user defined keys: ".@keys)
if($logger->is_debug());
    foreach my $kv (@keys){
    my($key,$value) = split(/=/,$kv);
    $logger->debug("Adding user defined key $key=$value in section
[$section]") if($logger->is_debug());
    my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
    if(!$ret){
        $cfg->newval($section, $delimeter.$key.$delimeter,$value);
        #$logger->logdie("Couldn't add key $delimeter $key
$delimeter=$value to section [$section]");
    }
    }
}

Jason

On 6/7/11 4:11 PM, Gary Xie wrote:



We managed to get the java environment setup and get past the earlier

problem, but now when I create a formatdb pipeline and run it, I get

this error.

 

more replace_config_keys.stderr

Died with 'Couldn't add key $; PIPELINEID $;=22 to section

[component]' at /opt/ergatis/bin/replace_config_keys.pl line 241

 

 

and if I look at the  formatdb.default.user.config

it doesn't contain the key, so it can't replace it.

 

 

But, if I look at  ergatis/docs/formatdb.config, it does have the keys

(with no values):

 

;the following keys are replaced at runtime by the invocation script

$;COMPONENT_CONFIG$; =

$;COMPONENT_XML$; =

$;PIPELINE_XML$; =

$;PIPELINEID$; =

 

 

I would guess it uses the formatdb.config to create the config file

for the pipeline, but don't understand why I'm losing the keys and

getting this error.

Can anyone help?

Thanks,

Gary

 

------------------------------------------------------------------------------

EditLive Enterprise is the world's most technically advanced content

authoring tool. Experience the power of Track Changes, Inline Image

Editing and ensure content is compliant with Accessibility Checking.

http://p.sf.net/sfu/ephox-dev2dev

_______________________________________________

Ergatis-users mailing list

[hidden email]

https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

 

 

 

 

------------------------------------------------------------------------------ EditLive Enterprise is the world's most technically advanced content authoring tool. Experience the power of Track Changes, Inline Image Editing and ensure content is compliant with Accessibility Checking. http://p.sf.net/sfu/ephox-dev2dev_______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users

 

 

 


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

Aaron Buechlein
Hi Karthik,
This may be a silly question, but are you sure that the user running
apache has the same read/write permissions for the appropriate
directories as you do?
Thanks,

Aaron

Muthuraman, Karthik Narayan (K) wrote:

>
> Thanks Jason. Also, when I run the shell script “pipeline.32.run.sh”
> under workflows/scripts from my shell, it createdsthe log file, though
> with errors.
>
> But when ergatis runs it as apache, it does not create the log file at
> all.
>
> I have provided all the environment vars.
>
> Sincerely,
> Karthik Narayan
>
> *From:* Jason Kwan [mailto:[hidden email]]
> *Sent:* Tuesday, June 14, 2011 2:13 PM
> *To:* Muthuraman, Karthik Narayan (K)
> *Cc:* [hidden email]
> *Subject:* Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> Hi Karthik,
>
> You got past the "create output directory" and "create temp directory"
> stages so I can conclude that you probably have the right permissions
> for the output directory (do these get created)? Other than that there
> is not much to go on, apart from exactly the same error message that
> Gary faced. If I was faced with this situation, I would double check
> that I had changed replace_config_keys.pl in the install directory,
> rather than the directory that I had unpacked the ergatis source
> package (sudo find / -name "replace_config_keys.pl"), then change all
> of them just to be safe (and keep a backup of the original of course).
>
> Other than that, I can't think of anything. Perhaps the devs have some
> further thoughts?
>
> Jason
>
> On 6/14/11 11:42 AM, Muthuraman, Karthik Narayan (K) wrote:
>
> Hello Jason
>
> I looked at the thread, and patches proposed did not fix my problem.
>
> There are 2 params in the repace_config_keys.pl, that do not exist.
> They are:
>
> COMPONENT_XML=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_default/component.xml
> --
> output_conf=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_default/formatdb.default.final.config
>
> These two files are not created. Any thoughts ?
>
> Sincerely,
> Karthik Narayan
>
> *From:* Jason Kwan [mailto:[hidden email]]
> *Sent:* Tuesday, June 14, 2011 1:15 PM
> *To:* [hidden email]
> <mailto:[hidden email]>
> *Subject:* Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> Hi Karthik,
>
> This could be the bug that was described on this list on the 7th and
> 8th of June, please look back in the archives and let us know if it
> fixes it.
>
> Thanks
>
> Jason
>
>
> On 6/14/11 9:50 AM, Muthuraman, Karthik Narayan (K) wrote:
>
> Hello All
>
> Thanks for your responses.
>
> I have narrowed down the possible causes for the errors.
>
> When I run the shell script under workflows/scripts, it does not
> produce any output. But when I run the final command “RunWorkflow” in
> the script, it produces the log file. Meaning the environment
> variables created in the script are wrong. However when I open a new
> shell and try running the command again, it does not work. So, there
> is some environment variable that is working in the current shell I’m
> working on, and I don’t know what that is. The only environment
> variable I added to the apache conf was PERL%LIB.
>
> Even when I ran the RunWorkflow command manually, I got this error :
>
> [root@nextgen ergatis]# cat
> projects/maize/workflow/runtime/formatdb/15_default/replace_config_keys.stderr
>
> Died with 'Couldn't add key $; PIPELINEID $;=15 to section
> [component]' at ../ergatis/bin/replace_config_keys.pl line 241
>
> I’m just trying to run formatdb on a fasta file.
>
> Sincerely,
> Karthik Narayan
>
> *From:* Gary Xie [mailto:[hidden email]]
> *Sent:* Tuesday, June 14, 2011 11:04 AM
> *To:* Kevin Galens
> *Cc:* [hidden email]
> <mailto:[hidden email]>
> *Subject:* Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> Hi, all,
>
> We hit another glitch; It seems the code that runs the workflow is
> broken. I was hoping to wait for the new release. But we could post
> the issue here:
>
> It looks like whatever is creating the name of the file passed into
> the script that fails
>
> opt/ergatis/bin/create_file_iterator_list.pl <http://t.pl/> line 140.
>
> is not substituting the pipleine id into the file name:
>
> the command is:
>
> /opt/ergatis/bin/create_file_iterator_list --input_directory=
> --input_directory_extension=fsa
> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
> --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/*_default*/i1.list
> --timestamp=0
>
> and should be
>
> /opt/ergatis/bin/create_file_iterator_list --input_directory=
> --input_directory_extension=fsa
> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
> --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblastn/*10_default*/i1.list
> --timestamp=0
>
> I don't know where in the ergatis code this substitution should
> happen. Any insight?
>
> Thanks,
>
> Gary
>
> On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:
>
>
>
>
>
> Yes, this definitely is a bug. Just out of curiosity, what version of
> the module are you using? It would be great if you setup a ticket on
> Sourceforge so we can take a look at this issue. Include your proposed
> changes as well.
>
> http://sourceforge.net/tracker/?group_id=148765&atid=772583
> <http://sourceforge.net/tracker/?group_id=148765&atid=772583>
>
> Thank you for figuring this out.
>
> Kevin
>
> *From: *Jason Kwan <[hidden email] <mailto:[hidden email]>>
> *Organization: *University of Utah
> *Reply-To: *<[hidden email] <mailto:[hidden email]>>,
> <[hidden email]
> <mailto:[hidden email]>>
> *Date: *Tue, 07 Jun 2011 19:20:55 -0600
> *To: *Gary Xie <[hidden email] <mailto:[hidden email]>>
> *Cc: *"[hidden email]
> <mailto:[hidden email]>"
> <[hidden email]
> <mailto:[hidden email]>>
> *Subject: *Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> Hi Everyone,
>
> I wonder what the Devs think of the error that Gary and I encountered.
> I suspect maybe Config::Inifiles used to allow setval() on something
> that didn't previously exist, and if so this should count as a bug?
>
> Jason
>
> On 6/7/11 5:12 PM, Gary Xie wrote:
>
> Jason, Thanks a lot for your help and insight.
>
> Gary
>
> On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:
>
>
>
>
>
> Hi Gary,
>
> I'm not if this will be the answer to your problem, but I found in my
> notes that I did an additional thing. There is another section of
> replace_config_keys.pl at ~line 62 that looks like this:
>
> #Write the output location of this file as key $;COMPONENT_CONFIG$;
> my $ret =
> $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
> if(!$ret){
> $logger->logdie("Couldn't add key
> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
> section [component]");
> }
> &check_parameters($cfg,'component');
>
> Which I changed to this:
> #Write the output location of this file as key $;COMPONENT_CONFIG$;
> my $ret =
> $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
> if(!$ret){
> $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$options{'output_conf'});
> #$logger->logdie("Couldn't add key
> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
> section [component]");
> }
> &check_parameters($cfg,'component');
>
> However, I think I did this at the same time as the last suggestion,
> so I never saw the same error that you have.
>
> Jason
>
> On 6/7/11 4:45 PM, Gary Xie wrote:
>
> Thanks, Jason,
>
> Here are details on the error message I got.
>
> Gary
>
>
>     formatted source viewer:
>     /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.default.user.config
>
>
>       [interface]
>
>     * classification = file / indexing
>
>
>       [parameters]
>
>     * enter T for protein, F for nucleotide
>     * $;PROTEIN$; = F
>
>
>       [input]
>
>     * space-separated list
>     * $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna
>
>
>       [output]
>
>     * $;OUTPUT_TOKEN$; = default
>     * $;OUTPUT_DIRECTORY$; =
>       $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
>     * call it whatever you like
>     * $;DATABASE_NAME$; = nucdb
>
>
>       [component]
>
>     * $;COMPONENT_NAME$; = formatdb
>     * $;DESCRIPTION$; = none
>     * $;WORKFLOW_REPOSITORY$; =
>       $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$;OUTPUT_TOKEN$;
>     * $;PIPELINE_TOKEN$; = unnamed
>     * The version,revision,tag here is set by an interpolated CVS tag
>     * $;VERSION$; = 2.0
>     * $;RELEASE_TAG$; = $Name$
>     * $;REVISION$; = $Revision: 6918 $
>     * $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml
>
>
>       [include]
>
>     * $;PROJECT_CONFIG$; =
>       /usr/local/projects/gary/workflow/project.config
>       <http://seed.lanl.gov/ergatis/cgi/view_formatted_ini_source.cgi?file=/usr/local/projects/gary/workflow/project.config>
>
> On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:
>
>
>
>
>
> Hi Gary,
>
> I had a similar problem with that part part of replace_config_keys.pl
> (i.e. the add_keys subroutine). For me anyway, it turned out that the
> version of Config::Inifiles that I had gives an error when you try to do
> setval() on a value that doesn't already exist.
>
> So I changed the original sub from:
> sub add_keys{
> my($cfg,$section,@keys) = @_;
> $logger->debug("Adding user defined keys: ".@keys)
> if($logger->is_debug());
> foreach my $kv (@keys){
> my($key,$value) = split(/=/,$kv);
> $logger->debug("Adding user defined key $key=$value in section
> [$section]") if($logger->is_debug());
> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
> if(!$ret){
> $logger->logdie("Couldn't add key $delimeter $key
> $delimeter=$value to section [$section]");
> }
> }
> }
>
> to:
> sub add_keys{
> my($cfg,$section,@keys) = @_;
> $logger->debug("Adding user defined keys: ".@keys)
> if($logger->is_debug());
> foreach my $kv (@keys){
> my($key,$value) = split(/=/,$kv);
> $logger->debug("Adding user defined key $key=$value in section
> [$section]") if($logger->is_debug());
> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
> if(!$ret){
> $cfg->newval($section, $delimeter.$key.$delimeter,$value);
> #$logger->logdie("Couldn't add key $delimeter $key
> $delimeter=$value to section [$section]");
> }
> }
> }
>
> Jason
>
> On 6/7/11 4:11 PM, Gary Xie wrote:
>
>
>
> We managed to get the java environment setup and get past the earlier
>
>     problem, but now when I create a formatdb pipeline and run it, I get
>
>     this error.
>
>         more replace_config_keys.stderr
>
>     Died with 'Couldn't add key $; PIPELINEID $;=22 to section
>
>     [component]' at /opt/ergatis/bin/replace_config_keys.pl line 241
>
>     and if I look at the formatdb.default.user.config
>
>     it doesn't contain the key, so it can't replace it.
>
>     But, if I look at ergatis/docs/formatdb.config, it does have the keys
>
>     (with no values):
>
>     ;the following keys are replaced at runtime by the invocation script
>
>     $;COMPONENT_CONFIG$; =
>
>     $;COMPONENT_XML$; =
>
>     $;PIPELINE_XML$; =
>
>     $;PIPELINEID$; =
>
>     I would guess it uses the formatdb.config to create the config file
>
>     for the pipeline, but don't understand why I'm losing the keys and
>
>     getting this error.
>
>     Can anyone help?
>
>     Thanks,
>
>     Gary
>
>     ------------------------------------------------------------------------------
>
>     EditLive Enterprise is the world's most technically advanced content
>
>     authoring tool. Experience the power of Track Changes, Inline Image
>
>     Editing and ensure content is compliant with Accessibility Checking.
>
>     http://p.sf.net/sfu/ephox-dev2dev
>
>     _______________________________________________
>
>     Ergatis-users mailing list
>
>     [hidden email]
>     <mailto:[hidden email]>
>
>     https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
>
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> <mailto:[hidden email]>
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev_______________________________________________
> Ergatis-users mailing list [hidden email]
> <mailto:[hidden email]>
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> ------------------------------------------------------------------------
>
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>  

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

kmuthuraman
Hello Aaron
I have set permissions to -R 777 for the workflow_manager and ergatis
directories for now. I don't believe it's a permission issue.

When " /apps/ergatis/bin/replace_config_keys" is run, I get the error

"Died with 'Couldn't add key $; PIPELINEID $;=33 to section [component]'
at ../ergatis/bin/replace_config_keys.pl line 269"

Sincerely,
Karthik Narayan


-----Original Message-----
From: Aaron Buechlein [mailto:[hidden email]]
Sent: Tuesday, June 14, 2011 2:44 PM
To: [hidden email]
Subject: Re: [Ergatis-users] error message on the one-step workflow:
formatdb

Hi Karthik,
This may be a silly question, but are you sure that the user running
apache has the same read/write permissions for the appropriate
directories as you do?
Thanks,

Aaron

Muthuraman, Karthik Narayan (K) wrote:

>
> Thanks Jason. Also, when I run the shell script "pipeline.32.run.sh"
> under workflows/scripts from my shell, it createdsthe log file, though
> with errors.
>
> But when ergatis runs it as apache, it does not create the log file at
> all.
>
> I have provided all the environment vars.
>
> Sincerely,
> Karthik Narayan
>
> *From:* Jason Kwan [mailto:[hidden email]]
> *Sent:* Tuesday, June 14, 2011 2:13 PM
> *To:* Muthuraman, Karthik Narayan (K)
> *Cc:* [hidden email]
> *Subject:* Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> Hi Karthik,
>
> You got past the "create output directory" and "create temp directory"
> stages so I can conclude that you probably have the right permissions
> for the output directory (do these get created)? Other than that there
> is not much to go on, apart from exactly the same error message that
> Gary faced. If I was faced with this situation, I would double check
> that I had changed replace_config_keys.pl in the install directory,
> rather than the directory that I had unpacked the ergatis source
> package (sudo find / -name "replace_config_keys.pl"), then change all
> of them just to be safe (and keep a backup of the original of course).
>
> Other than that, I can't think of anything. Perhaps the devs have some
> further thoughts?
>
> Jason
>
> On 6/14/11 11:42 AM, Muthuraman, Karthik Narayan (K) wrote:
>
> Hello Jason
>
> I looked at the thread, and patches proposed did not fix my problem.
>
> There are 2 params in the repace_config_keys.pl, that do not exist.
> They are:
>
>
COMPONENT_XML=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_
default/component.xml
> --
>
output_conf=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_de
fault/formatdb.default.final.config

>
> These two files are not created. Any thoughts ?
>
> Sincerely,
> Karthik Narayan
>
> *From:* Jason Kwan [mailto:[hidden email]]
> *Sent:* Tuesday, June 14, 2011 1:15 PM
> *To:* [hidden email]
> <mailto:[hidden email]>
> *Subject:* Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> Hi Karthik,
>
> This could be the bug that was described on this list on the 7th and
> 8th of June, please look back in the archives and let us know if it
> fixes it.
>
> Thanks
>
> Jason
>
>
> On 6/14/11 9:50 AM, Muthuraman, Karthik Narayan (K) wrote:
>
> Hello All
>
> Thanks for your responses.
>
> I have narrowed down the possible causes for the errors.
>
> When I run the shell script under workflows/scripts, it does not
> produce any output. But when I run the final command "RunWorkflow" in
> the script, it produces the log file. Meaning the environment
> variables created in the script are wrong. However when I open a new
> shell and try running the command again, it does not work. So, there
> is some environment variable that is working in the current shell I'm
> working on, and I don't know what that is. The only environment
> variable I added to the apache conf was PERL%LIB.
>
> Even when I ran the RunWorkflow command manually, I got this error :
>
> [root@nextgen ergatis]# cat
>
projects/maize/workflow/runtime/formatdb/15_default/replace_config_keys.
stderr

>
> Died with 'Couldn't add key $; PIPELINEID $;=15 to section
> [component]' at ../ergatis/bin/replace_config_keys.pl line 241
>
> I'm just trying to run formatdb on a fasta file.
>
> Sincerely,
> Karthik Narayan
>
> *From:* Gary Xie [mailto:[hidden email]]
> *Sent:* Tuesday, June 14, 2011 11:04 AM
> *To:* Kevin Galens
> *Cc:* [hidden email]
> <mailto:[hidden email]>
> *Subject:* Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> Hi, all,
>
> We hit another glitch; It seems the code that runs the workflow is
> broken. I was hoping to wait for the new release. But we could post
> the issue here:
>
> It looks like whatever is creating the name of the file passed into
> the script that fails
>
> opt/ergatis/bin/create_file_iterator_list.pl <http://t.pl/> line 140.
>
> is not substituting the pipleine id into the file name:
>
> the command is:
>
> /opt/ergatis/bin/create_file_iterator_list --input_directory=
> --input_directory_extension=fsa
> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
>
--output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblast
n/*_default*/i1.list
> --timestamp=0
>
> and should be
>
> /opt/ergatis/bin/create_file_iterator_list --input_directory=
> --input_directory_extension=fsa
> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
>
--output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblast
n/*10_default*/i1.list

> --timestamp=0
>
> I don't know where in the ergatis code this substitution should
> happen. Any insight?
>
> Thanks,
>
> Gary
>
> On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:
>
>
>
>
>
> Yes, this definitely is a bug. Just out of curiosity, what version of
> the module are you using? It would be great if you setup a ticket on
> Sourceforge so we can take a look at this issue. Include your proposed
> changes as well.
>
> http://sourceforge.net/tracker/?group_id=148765&atid=772583
> <http://sourceforge.net/tracker/?group_id=148765&atid=772583>
>
> Thank you for figuring this out.
>
> Kevin
>
> *From: *Jason Kwan <[hidden email] <mailto:[hidden email]>>
> *Organization: *University of Utah
> *Reply-To: *<[hidden email] <mailto:[hidden email]>>,
> <[hidden email]
> <mailto:[hidden email]>>
> *Date: *Tue, 07 Jun 2011 19:20:55 -0600
> *To: *Gary Xie <[hidden email] <mailto:[hidden email]>>
> *Cc: *"[hidden email]
> <mailto:[hidden email]>"
> <[hidden email]
> <mailto:[hidden email]>>
> *Subject: *Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> Hi Everyone,
>
> I wonder what the Devs think of the error that Gary and I encountered.
> I suspect maybe Config::Inifiles used to allow setval() on something
> that didn't previously exist, and if so this should count as a bug?
>
> Jason
>
> On 6/7/11 5:12 PM, Gary Xie wrote:
>
> Jason, Thanks a lot for your help and insight.
>
> Gary
>
> On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:
>
>
>
>
>
> Hi Gary,
>
> I'm not if this will be the answer to your problem, but I found in my
> notes that I did an additional thing. There is another section of
> replace_config_keys.pl at ~line 62 that looks like this:
>
> #Write the output location of this file as key $;COMPONENT_CONFIG$;
> my $ret =
>
$cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
ns{'output_conf'});

> if(!$ret){
> $logger->logdie("Couldn't add key
> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
> section [component]");
> }
> &check_parameters($cfg,'component');
>
> Which I changed to this:
> #Write the output location of this file as key $;COMPONENT_CONFIG$;
> my $ret =
>
$cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
ns{'output_conf'});
> if(!$ret){
>
$cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
ns{'output_conf'});

> #$logger->logdie("Couldn't add key
> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
> section [component]");
> }
> &check_parameters($cfg,'component');
>
> However, I think I did this at the same time as the last suggestion,
> so I never saw the same error that you have.
>
> Jason
>
> On 6/7/11 4:45 PM, Gary Xie wrote:
>
> Thanks, Jason,
>
> Here are details on the error message I got.
>
> Gary
>
>
>     formatted source viewer:
>
/usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.d
efault.user.config

>
>
>       [interface]
>
>     * classification = file / indexing
>
>
>       [parameters]
>
>     * enter T for protein, F for nucleotide
>     * $;PROTEIN$; = F
>
>
>       [input]
>
>     * space-separated list
>     * $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna
>
>
>       [output]
>
>     * $;OUTPUT_TOKEN$; = default
>     * $;OUTPUT_DIRECTORY$; =
>
$;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_
$;OUTPUT_TOKEN$;

>     * call it whatever you like
>     * $;DATABASE_NAME$; = nucdb
>
>
>       [component]
>
>     * $;COMPONENT_NAME$; = formatdb
>     * $;DESCRIPTION$; = none
>     * $;WORKFLOW_REPOSITORY$; =
>
$;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$
;OUTPUT_TOKEN$;

>     * $;PIPELINE_TOKEN$; = unnamed
>     * The version,revision,tag here is set by an interpolated CVS tag
>     * $;VERSION$; = 2.0
>     * $;RELEASE_TAG$; = $Name$
>     * $;REVISION$; = $Revision: 6918 $
>     * $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml
>
>
>       [include]
>
>     * $;PROJECT_CONFIG$; =
>       /usr/local/projects/gary/workflow/project.config
>
<http://seed.lanl.gov/ergatis/cgi/view_formatted_ini_source.cgi?file=/us
r/local/projects/gary/workflow/project.config>

>
> On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:
>
>
>
>
>
> Hi Gary,
>
> I had a similar problem with that part part of replace_config_keys.pl
> (i.e. the add_keys subroutine). For me anyway, it turned out that the
> version of Config::Inifiles that I had gives an error when you try to
do

> setval() on a value that doesn't already exist.
>
> So I changed the original sub from:
> sub add_keys{
> my($cfg,$section,@keys) = @_;
> $logger->debug("Adding user defined keys: ".@keys)
> if($logger->is_debug());
> foreach my $kv (@keys){
> my($key,$value) = split(/=/,$kv);
> $logger->debug("Adding user defined key $key=$value in section
> [$section]") if($logger->is_debug());
> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
> if(!$ret){
> $logger->logdie("Couldn't add key $delimeter $key
> $delimeter=$value to section [$section]");
> }
> }
> }
>
> to:
> sub add_keys{
> my($cfg,$section,@keys) = @_;
> $logger->debug("Adding user defined keys: ".@keys)
> if($logger->is_debug());
> foreach my $kv (@keys){
> my($key,$value) = split(/=/,$kv);
> $logger->debug("Adding user defined key $key=$value in section
> [$section]") if($logger->is_debug());
> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
> if(!$ret){
> $cfg->newval($section, $delimeter.$key.$delimeter,$value);
> #$logger->logdie("Couldn't add key $delimeter $key
> $delimeter=$value to section [$section]");
> }
> }
> }
>
> Jason
>
> On 6/7/11 4:11 PM, Gary Xie wrote:
>
>
>
> We managed to get the java environment setup and get past the earlier
>
>     problem, but now when I create a formatdb pipeline and run it, I
get

>
>     this error.
>
>         more replace_config_keys.stderr
>
>     Died with 'Couldn't add key $; PIPELINEID $;=22 to section
>
>     [component]' at /opt/ergatis/bin/replace_config_keys.pl line 241
>
>     and if I look at the formatdb.default.user.config
>
>     it doesn't contain the key, so it can't replace it.
>
>     But, if I look at ergatis/docs/formatdb.config, it does have the
keys
>
>     (with no values):
>
>     ;the following keys are replaced at runtime by the invocation
script

>
>     $;COMPONENT_CONFIG$; =
>
>     $;COMPONENT_XML$; =
>
>     $;PIPELINE_XML$; =
>
>     $;PIPELINEID$; =
>
>     I would guess it uses the formatdb.config to create the config
file

>
>     for the pipeline, but don't understand why I'm losing the keys and
>
>     getting this error.
>
>     Can anyone help?
>
>     Thanks,
>
>     Gary
>
>
------------------------------------------------------------------------
------
>
>     EditLive Enterprise is the world's most technically advanced
content
>
>     authoring tool. Experience the power of Track Changes, Inline
Image
>
>     Editing and ensure content is compliant with Accessibility
Checking.

>
>     http://p.sf.net/sfu/ephox-dev2dev
>
>     _______________________________________________
>
>     Ergatis-users mailing list
>
>     [hidden email]
>     <mailto:[hidden email]>
>
>     https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
>
>
>
------------------------------------------------------------------------
------

> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> <mailto:[hidden email]>
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
>
------------------------------------------------------------------------
------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
>
http://p.sf.net/sfu/ephox-dev2dev_______________________________________
________
> Ergatis-users mailing list [hidden email]
> <mailto:[hidden email]>
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
>
------------------------------------------------------------------------
>
>
------------------------------------------------------------------------
------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
>
------------------------------------------------------------------------
>
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>  

------------------------------------------------------------------------
------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

Jason Kwan
Hi Karthik,

I noticed that before you had the same error message but with line 241.  
Now it is line 269.  Now, in the version of replace_config_keys.pl that
I have (suitable altered, as mentioned before), has only 260 lines.  I
suspect it is one of two possibilities - 1. you have a different version
of ergatis/the script (I have ergatis v2r13b2), or 2. maybe you added
sections to the script rather than substituting?  I have attached the
version that I have so you can compare.

Thanks, and please let us know if this doesn't fix it.

Jason


On 6/14/11 12:52 PM, Muthuraman, Karthik Narayan (K) wrote:

> Hello Aaron
> I have set permissions to -R 777 for the workflow_manager and ergatis
> directories for now. I don't believe it's a permission issue.
>
> When " /apps/ergatis/bin/replace_config_keys" is run, I get the error
>
> "Died with 'Couldn't add key $; PIPELINEID $;=33 to section [component]'
> at ../ergatis/bin/replace_config_keys.pl line 269"
>
> Sincerely,
> Karthik Narayan
>
>
> -----Original Message-----
> From: Aaron Buechlein [mailto:[hidden email]]
> Sent: Tuesday, June 14, 2011 2:44 PM
> To: [hidden email]
> Subject: Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> Hi Karthik,
> This may be a silly question, but are you sure that the user running
> apache has the same read/write permissions for the appropriate
> directories as you do?
> Thanks,
>
> Aaron
>
> Muthuraman, Karthik Narayan (K) wrote:
>> Thanks Jason. Also, when I run the shell script "pipeline.32.run.sh"
>> under workflows/scripts from my shell, it createdsthe log file, though
>> with errors.
>>
>> But when ergatis runs it as apache, it does not create the log file at
>> all.
>>
>> I have provided all the environment vars.
>>
>> Sincerely,
>> Karthik Narayan
>>
>> *From:* Jason Kwan [mailto:[hidden email]]
>> *Sent:* Tuesday, June 14, 2011 2:13 PM
>> *To:* Muthuraman, Karthik Narayan (K)
>> *Cc:* [hidden email]
>> *Subject:* Re: [Ergatis-users] error message on the one-step workflow:
>> formatdb
>>
>> Hi Karthik,
>>
>> You got past the "create output directory" and "create temp directory"
>> stages so I can conclude that you probably have the right permissions
>> for the output directory (do these get created)? Other than that there
>> is not much to go on, apart from exactly the same error message that
>> Gary faced. If I was faced with this situation, I would double check
>> that I had changed replace_config_keys.pl in the install directory,
>> rather than the directory that I had unpacked the ergatis source
>> package (sudo find / -name "replace_config_keys.pl"), then change all
>> of them just to be safe (and keep a backup of the original of course).
>>
>> Other than that, I can't think of anything. Perhaps the devs have some
>> further thoughts?
>>
>> Jason
>>
>> On 6/14/11 11:42 AM, Muthuraman, Karthik Narayan (K) wrote:
>>
>> Hello Jason
>>
>> I looked at the thread, and patches proposed did not fix my problem.
>>
>> There are 2 params in the repace_config_keys.pl, that do not exist.
>> They are:
>>
>>
> COMPONENT_XML=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_
> default/component.xml
>> --
>>
> output_conf=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_de
> fault/formatdb.default.final.config
>> These two files are not created. Any thoughts ?
>>
>> Sincerely,
>> Karthik Narayan
>>
>> *From:* Jason Kwan [mailto:[hidden email]]
>> *Sent:* Tuesday, June 14, 2011 1:15 PM
>> *To:* [hidden email]
>> <mailto:[hidden email]>
>> *Subject:* Re: [Ergatis-users] error message on the one-step workflow:
>> formatdb
>>
>> Hi Karthik,
>>
>> This could be the bug that was described on this list on the 7th and
>> 8th of June, please look back in the archives and let us know if it
>> fixes it.
>>
>> Thanks
>>
>> Jason
>>
>>
>> On 6/14/11 9:50 AM, Muthuraman, Karthik Narayan (K) wrote:
>>
>> Hello All
>>
>> Thanks for your responses.
>>
>> I have narrowed down the possible causes for the errors.
>>
>> When I run the shell script under workflows/scripts, it does not
>> produce any output. But when I run the final command "RunWorkflow" in
>> the script, it produces the log file. Meaning the environment
>> variables created in the script are wrong. However when I open a new
>> shell and try running the command again, it does not work. So, there
>> is some environment variable that is working in the current shell I'm
>> working on, and I don't know what that is. The only environment
>> variable I added to the apache conf was PERL%LIB.
>>
>> Even when I ran the RunWorkflow command manually, I got this error :
>>
>> [root@nextgen ergatis]# cat
>>
> projects/maize/workflow/runtime/formatdb/15_default/replace_config_keys.
> stderr
>> Died with 'Couldn't add key $; PIPELINEID $;=15 to section
>> [component]' at ../ergatis/bin/replace_config_keys.pl line 241
>>
>> I'm just trying to run formatdb on a fasta file.
>>
>> Sincerely,
>> Karthik Narayan
>>
>> *From:* Gary Xie [mailto:[hidden email]]
>> *Sent:* Tuesday, June 14, 2011 11:04 AM
>> *To:* Kevin Galens
>> *Cc:* [hidden email]
>> <mailto:[hidden email]>
>> *Subject:* Re: [Ergatis-users] error message on the one-step workflow:
>> formatdb
>>
>> Hi, all,
>>
>> We hit another glitch; It seems the code that runs the workflow is
>> broken. I was hoping to wait for the new release. But we could post
>> the issue here:
>>
>> It looks like whatever is creating the name of the file passed into
>> the script that fails
>>
>> opt/ergatis/bin/create_file_iterator_list.pl<http://t.pl/>  line 140.
>>
>> is not substituting the pipleine id into the file name:
>>
>> the command is:
>>
>> /opt/ergatis/bin/create_file_iterator_list --input_directory=
>> --input_directory_extension=fsa
>> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
>>
> --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblast
> n/*_default*/i1.list
>> --timestamp=0
>>
>> and should be
>>
>> /opt/ergatis/bin/create_file_iterator_list --input_directory=
>> --input_directory_extension=fsa
>> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
>>
> --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblast
> n/*10_default*/i1.list
>> --timestamp=0
>>
>> I don't know where in the ergatis code this substitution should
>> happen. Any insight?
>>
>> Thanks,
>>
>> Gary
>>
>> On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:
>>
>>
>>
>>
>>
>> Yes, this definitely is a bug. Just out of curiosity, what version of
>> the module are you using? It would be great if you setup a ticket on
>> Sourceforge so we can take a look at this issue. Include your proposed
>> changes as well.
>>
>> http://sourceforge.net/tracker/?group_id=148765&atid=772583
>> <http://sourceforge.net/tracker/?group_id=148765&atid=772583>
>>
>> Thank you for figuring this out.
>>
>> Kevin
>>
>> *From: *Jason Kwan<[hidden email]<mailto:[hidden email]>>
>> *Organization: *University of Utah
>> *Reply-To: *<[hidden email]<mailto:[hidden email]>>,
>> <[hidden email]
>> <mailto:[hidden email]>>
>> *Date: *Tue, 07 Jun 2011 19:20:55 -0600
>> *To: *Gary Xie<[hidden email]<mailto:[hidden email]>>
>> *Cc: *"[hidden email]
>> <mailto:[hidden email]>"
>> <[hidden email]
>> <mailto:[hidden email]>>
>> *Subject: *Re: [Ergatis-users] error message on the one-step workflow:
>> formatdb
>>
>> Hi Everyone,
>>
>> I wonder what the Devs think of the error that Gary and I encountered.
>> I suspect maybe Config::Inifiles used to allow setval() on something
>> that didn't previously exist, and if so this should count as a bug?
>>
>> Jason
>>
>> On 6/7/11 5:12 PM, Gary Xie wrote:
>>
>> Jason, Thanks a lot for your help and insight.
>>
>> Gary
>>
>> On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:
>>
>>
>>
>>
>>
>> Hi Gary,
>>
>> I'm not if this will be the answer to your problem, but I found in my
>> notes that I did an additional thing. There is another section of
>> replace_config_keys.pl at ~line 62 that looks like this:
>>
>> #Write the output location of this file as key $;COMPONENT_CONFIG$;
>> my $ret =
>>
> $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
> ns{'output_conf'});
>> if(!$ret){
>> $logger->logdie("Couldn't add key
>> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
>> section [component]");
>> }
>> &check_parameters($cfg,'component');
>>
>> Which I changed to this:
>> #Write the output location of this file as key $;COMPONENT_CONFIG$;
>> my $ret =
>>
> $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
> ns{'output_conf'});
>> if(!$ret){
>>
> $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
> ns{'output_conf'});
>> #$logger->logdie("Couldn't add key
>> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
>> section [component]");
>> }
>> &check_parameters($cfg,'component');
>>
>> However, I think I did this at the same time as the last suggestion,
>> so I never saw the same error that you have.
>>
>> Jason
>>
>> On 6/7/11 4:45 PM, Gary Xie wrote:
>>
>> Thanks, Jason,
>>
>> Here are details on the error message I got.
>>
>> Gary
>>
>>
>>      formatted source viewer:
>>
> /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.d
> efault.user.config
>>
>>        [interface]
>>
>>      * classification = file / indexing
>>
>>
>>        [parameters]
>>
>>      * enter T for protein, F for nucleotide
>>      * $;PROTEIN$; = F
>>
>>
>>        [input]
>>
>>      * space-separated list
>>      * $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna
>>
>>
>>        [output]
>>
>>      * $;OUTPUT_TOKEN$; = default
>>      * $;OUTPUT_DIRECTORY$; =
>>
> $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_
> $;OUTPUT_TOKEN$;
>>      * call it whatever you like
>>      * $;DATABASE_NAME$; = nucdb
>>
>>
>>        [component]
>>
>>      * $;COMPONENT_NAME$; = formatdb
>>      * $;DESCRIPTION$; = none
>>      * $;WORKFLOW_REPOSITORY$; =
>>
> $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$
> ;OUTPUT_TOKEN$;
>>      * $;PIPELINE_TOKEN$; = unnamed
>>      * The version,revision,tag here is set by an interpolated CVS tag
>>      * $;VERSION$; = 2.0
>>      * $;RELEASE_TAG$; = $Name$
>>      * $;REVISION$; = $Revision: 6918 $
>>      * $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml
>>
>>
>>        [include]
>>
>>      * $;PROJECT_CONFIG$; =
>>        /usr/local/projects/gary/workflow/project.config
>>
> <http://seed.lanl.gov/ergatis/cgi/view_formatted_ini_source.cgi?file=/us
> r/local/projects/gary/workflow/project.config>
>> On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:
>>
>>
>>
>>
>>
>> Hi Gary,
>>
>> I had a similar problem with that part part of replace_config_keys.pl
>> (i.e. the add_keys subroutine). For me anyway, it turned out that the
>> version of Config::Inifiles that I had gives an error when you try to
> do
>> setval() on a value that doesn't already exist.
>>
>> So I changed the original sub from:
>> sub add_keys{
>> my($cfg,$section,@keys) = @_;
>> $logger->debug("Adding user defined keys: ".@keys)
>> if($logger->is_debug());
>> foreach my $kv (@keys){
>> my($key,$value) = split(/=/,$kv);
>> $logger->debug("Adding user defined key $key=$value in section
>> [$section]") if($logger->is_debug());
>> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
>> if(!$ret){
>> $logger->logdie("Couldn't add key $delimeter $key
>> $delimeter=$value to section [$section]");
>> }
>> }
>> }
>>
>> to:
>> sub add_keys{
>> my($cfg,$section,@keys) = @_;
>> $logger->debug("Adding user defined keys: ".@keys)
>> if($logger->is_debug());
>> foreach my $kv (@keys){
>> my($key,$value) = split(/=/,$kv);
>> $logger->debug("Adding user defined key $key=$value in section
>> [$section]") if($logger->is_debug());
>> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
>> if(!$ret){
>> $cfg->newval($section, $delimeter.$key.$delimeter,$value);
>> #$logger->logdie("Couldn't add key $delimeter $key
>> $delimeter=$value to section [$section]");
>> }
>> }
>> }
>>
>> Jason
>>
>> On 6/7/11 4:11 PM, Gary Xie wrote:
>>
>>
>>
>> We managed to get the java environment setup and get past the earlier
>>
>>      problem, but now when I create a formatdb pipeline and run it, I
> get
>>      this error.
>>
>>          more replace_config_keys.stderr
>>
>>      Died with 'Couldn't add key $; PIPELINEID $;=22 to section
>>
>>      [component]' at /opt/ergatis/bin/replace_config_keys.pl line 241
>>
>>      and if I look at the formatdb.default.user.config
>>
>>      it doesn't contain the key, so it can't replace it.
>>
>>      But, if I look at ergatis/docs/formatdb.config, it does have the
> keys
>>      (with no values):
>>
>>      ;the following keys are replaced at runtime by the invocation
> script
>>      $;COMPONENT_CONFIG$; =
>>
>>      $;COMPONENT_XML$; =
>>
>>      $;PIPELINE_XML$; =
>>
>>      $;PIPELINEID$; =
>>
>>      I would guess it uses the formatdb.config to create the config
> file
>>      for the pipeline, but don't understand why I'm losing the keys and
>>
>>      getting this error.
>>
>>      Can anyone help?
>>
>>      Thanks,
>>
>>      Gary
>>
>>
> ------------------------------------------------------------------------
> ------
>>      EditLive Enterprise is the world's most technically advanced
> content
>>      authoring tool. Experience the power of Track Changes, Inline
> Image
>>      Editing and ensure content is compliant with Accessibility
> Checking.
>>      http://p.sf.net/sfu/ephox-dev2dev
>>
>>      _______________________________________________
>>
>>      Ergatis-users mailing list
>>
>>      [hidden email]
>>      <mailto:[hidden email]>
>>
>>      https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>>
>>
>>
> ------------------------------------------------------------------------
> ------
>> EditLive Enterprise is the world's most technically advanced content
>> authoring tool. Experience the power of Track Changes, Inline Image
>> Editing and ensure content is compliant with Accessibility Checking.
>> http://p.sf.net/sfu/ephox-dev2dev
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> <mailto:[hidden email]>
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>>
> ------------------------------------------------------------------------
> ------
>> EditLive Enterprise is the world's most technically advanced content
>> authoring tool. Experience the power of Track Changes, Inline Image
>> Editing and ensure content is compliant with Accessibility Checking.
>>
> http://p.sf.net/sfu/ephox-dev2dev_______________________________________
> ________
>> Ergatis-users mailing list [hidden email]
>> <mailto:[hidden email]>
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>>
> ------------------------------------------------------------------------
>>
> ------------------------------------------------------------------------
> ------
>> EditLive Enterprise is the world's most technically advanced content
>> authoring tool. Experience the power of Track Changes, Inline Image
>> Editing and ensure content is compliant with Accessibility Checking.
>> http://p.sf.net/sfu/ephox-dev2dev
>>
> ------------------------------------------------------------------------
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
> ------------------------------------------------------------------------
> ------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

replace_config_keys.pl (9K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

kmuthuraman
Thanks Jason. I had commented the lines out and hence the difference in
line numbers. I had also made a mistake while copying the patch, which I
fixed now.

I get this error now:
Died with 'Bad key $;INPUT_FILES$; in template file' at
/apps/ergatis/bin/replace_template_keys.pl line 282

In the interface, I gave the full path to my fasta file in "Input
Files". But in workflow/runtime/pipeline/42/formatdb.default.config, the
fasta file is assigned to " $;INPUT_FILE_LIST$;=/apps/ergatis/test.faa",
and there is no $;INPUT_FILES$;

Which file do I have to change ?

Sincerely,
Karthik Narayan


-----Original Message-----
From: Jason Kwan [mailto:[hidden email]]
Sent: Tuesday, June 14, 2011 3:14 PM
To: [hidden email]
Subject: Re: [Ergatis-users] error message on the one-step workflow:
formatdb

The Dow Chemical Company's NEMX filtering software is automatically
removing the following e-mail attachments from incoming messages as part
of Dow's virus prevention program:



     .bas, .bat, .bin, .chm, .cmd, .com, .cpl, .csc, .dll, .dbx, .exe,
.hlp, .hta, .inf, .ins, .js, .jse,

     .lnk, .msc, .msi, .msp, .mst, .nch, .pif .pl, .reg, .scr, .sct,
.shb, .shs, .wsf, .wsh, .vb, .vbs
     .vbe, .vxd, .wsc


For clarification or more information, click here:

 
http:////////////////manage-info.intranet.dow.com////////Information_Sec
urity////////FAQ////////IS(FAQ)attachments.htmHi Karthik,

I noticed that before you had the same error message but with line 241.

Now it is line 269.  Now, in the version of replace_config_keys.pl that
I have (suitable altered, as mentioned before), has only 260 lines.  I
suspect it is one of two possibilities - 1. you have a different version

of ergatis/the script (I have ergatis v2r13b2), or 2. maybe you added
sections to the script rather than substituting?  I have attached the
version that I have so you can compare.

Thanks, and please let us know if this doesn't fix it.

Jason


On 6/14/11 12:52 PM, Muthuraman, Karthik Narayan (K) wrote:
> Hello Aaron
> I have set permissions to -R 777 for the workflow_manager and ergatis
> directories for now. I don't believe it's a permission issue.
>
> When " /apps/ergatis/bin/replace_config_keys" is run, I get the error
>
> "Died with 'Couldn't add key $; PIPELINEID $;=33 to section
[component]'

> at ../ergatis/bin/replace_config_keys.pl line 269"
>
> Sincerely,
> Karthik Narayan
>
>
> -----Original Message-----
> From: Aaron Buechlein [mailto:[hidden email]]
> Sent: Tuesday, June 14, 2011 2:44 PM
> To: [hidden email]
> Subject: Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> Hi Karthik,
> This may be a silly question, but are you sure that the user running
> apache has the same read/write permissions for the appropriate
> directories as you do?
> Thanks,
>
> Aaron
>
> Muthuraman, Karthik Narayan (K) wrote:
>> Thanks Jason. Also, when I run the shell script "pipeline.32.run.sh"
>> under workflows/scripts from my shell, it createdsthe log file,
though
>> with errors.
>>
>> But when ergatis runs it as apache, it does not create the log file
at

>> all.
>>
>> I have provided all the environment vars.
>>
>> Sincerely,
>> Karthik Narayan
>>
>> *From:* Jason Kwan [mailto:[hidden email]]
>> *Sent:* Tuesday, June 14, 2011 2:13 PM
>> *To:* Muthuraman, Karthik Narayan (K)
>> *Cc:* [hidden email]
>> *Subject:* Re: [Ergatis-users] error message on the one-step
workflow:
>> formatdb
>>
>> Hi Karthik,
>>
>> You got past the "create output directory" and "create temp
directory"
>> stages so I can conclude that you probably have the right permissions
>> for the output directory (do these get created)? Other than that
there
>> is not much to go on, apart from exactly the same error message that
>> Gary faced. If I was faced with this situation, I would double check
>> that I had changed replace_config_keys.pl in the install directory,
>> rather than the directory that I had unpacked the ergatis source
>> package (sudo find / -name "replace_config_keys.pl"), then change all
>> of them just to be safe (and keep a backup of the original of
course).
>>
>> Other than that, I can't think of anything. Perhaps the devs have
some

>> further thoughts?
>>
>> Jason
>>
>> On 6/14/11 11:42 AM, Muthuraman, Karthik Narayan (K) wrote:
>>
>> Hello Jason
>>
>> I looked at the thread, and patches proposed did not fix my problem.
>>
>> There are 2 params in the repace_config_keys.pl, that do not exist.
>> They are:
>>
>>
>
COMPONENT_XML=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_
> default/component.xml
>> --
>>
>
output_conf=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_de

> fault/formatdb.default.final.config
>> These two files are not created. Any thoughts ?
>>
>> Sincerely,
>> Karthik Narayan
>>
>> *From:* Jason Kwan [mailto:[hidden email]]
>> *Sent:* Tuesday, June 14, 2011 1:15 PM
>> *To:* [hidden email]
>> <mailto:[hidden email]>
>> *Subject:* Re: [Ergatis-users] error message on the one-step
workflow:

>> formatdb
>>
>> Hi Karthik,
>>
>> This could be the bug that was described on this list on the 7th and
>> 8th of June, please look back in the archives and let us know if it
>> fixes it.
>>
>> Thanks
>>
>> Jason
>>
>>
>> On 6/14/11 9:50 AM, Muthuraman, Karthik Narayan (K) wrote:
>>
>> Hello All
>>
>> Thanks for your responses.
>>
>> I have narrowed down the possible causes for the errors.
>>
>> When I run the shell script under workflows/scripts, it does not
>> produce any output. But when I run the final command "RunWorkflow" in
>> the script, it produces the log file. Meaning the environment
>> variables created in the script are wrong. However when I open a new
>> shell and try running the command again, it does not work. So, there
>> is some environment variable that is working in the current shell I'm
>> working on, and I don't know what that is. The only environment
>> variable I added to the apache conf was PERL%LIB.
>>
>> Even when I ran the RunWorkflow command manually, I got this error :
>>
>> [root@nextgen ergatis]# cat
>>
>
projects/maize/workflow/runtime/formatdb/15_default/replace_config_keys.

> stderr
>> Died with 'Couldn't add key $; PIPELINEID $;=15 to section
>> [component]' at ../ergatis/bin/replace_config_keys.pl line 241
>>
>> I'm just trying to run formatdb on a fasta file.
>>
>> Sincerely,
>> Karthik Narayan
>>
>> *From:* Gary Xie [mailto:[hidden email]]
>> *Sent:* Tuesday, June 14, 2011 11:04 AM
>> *To:* Kevin Galens
>> *Cc:* [hidden email]
>> <mailto:[hidden email]>
>> *Subject:* Re: [Ergatis-users] error message on the one-step
workflow:

>> formatdb
>>
>> Hi, all,
>>
>> We hit another glitch; It seems the code that runs the workflow is
>> broken. I was hoping to wait for the new release. But we could post
>> the issue here:
>>
>> It looks like whatever is creating the name of the file passed into
>> the script that fails
>>
>> opt/ergatis/bin/create_file_iterator_list.pl<http://t.pl/>  line 140.
>>
>> is not substituting the pipleine id into the file name:
>>
>> the command is:
>>
>> /opt/ergatis/bin/create_file_iterator_list --input_directory=
>> --input_directory_extension=fsa
>> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
>>
>
--output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblast

> n/*_default*/i1.list
>> --timestamp=0
>>
>> and should be
>>
>> /opt/ergatis/bin/create_file_iterator_list --input_directory=
>> --input_directory_extension=fsa
>> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
>>
>
--output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblast

> n/*10_default*/i1.list
>> --timestamp=0
>>
>> I don't know where in the ergatis code this substitution should
>> happen. Any insight?
>>
>> Thanks,
>>
>> Gary
>>
>> On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:
>>
>>
>>
>>
>>
>> Yes, this definitely is a bug. Just out of curiosity, what version of
>> the module are you using? It would be great if you setup a ticket on
>> Sourceforge so we can take a look at this issue. Include your
proposed

>> changes as well.
>>
>> http://sourceforge.net/tracker/?group_id=148765&atid=772583
>> <http://sourceforge.net/tracker/?group_id=148765&atid=772583>
>>
>> Thank you for figuring this out.
>>
>> Kevin
>>
>> *From: *Jason Kwan<[hidden email]<mailto:[hidden email]>>
>> *Organization: *University of Utah
>> *Reply-To: *<[hidden email]<mailto:[hidden email]>>,
>> <[hidden email]
>> <mailto:[hidden email]>>
>> *Date: *Tue, 07 Jun 2011 19:20:55 -0600
>> *To: *Gary Xie<[hidden email]<mailto:[hidden email]>>
>> *Cc: *"[hidden email]
>> <mailto:[hidden email]>"
>> <[hidden email]
>> <mailto:[hidden email]>>
>> *Subject: *Re: [Ergatis-users] error message on the one-step
workflow:
>> formatdb
>>
>> Hi Everyone,
>>
>> I wonder what the Devs think of the error that Gary and I
encountered.

>> I suspect maybe Config::Inifiles used to allow setval() on something
>> that didn't previously exist, and if so this should count as a bug?
>>
>> Jason
>>
>> On 6/7/11 5:12 PM, Gary Xie wrote:
>>
>> Jason, Thanks a lot for your help and insight.
>>
>> Gary
>>
>> On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:
>>
>>
>>
>>
>>
>> Hi Gary,
>>
>> I'm not if this will be the answer to your problem, but I found in my
>> notes that I did an additional thing. There is another section of
>> replace_config_keys.pl at ~line 62 that looks like this:
>>
>> #Write the output location of this file as key $;COMPONENT_CONFIG$;
>> my $ret =
>>
>
$cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio

> ns{'output_conf'});
>> if(!$ret){
>> $logger->logdie("Couldn't add key
>> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
>> section [component]");
>> }
>> &check_parameters($cfg,'component');
>>
>> Which I changed to this:
>> #Write the output location of this file as key $;COMPONENT_CONFIG$;
>> my $ret =
>>
>
$cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
> ns{'output_conf'});
>> if(!$ret){
>>
>
$cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio

> ns{'output_conf'});
>> #$logger->logdie("Couldn't add key
>> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
>> section [component]");
>> }
>> &check_parameters($cfg,'component');
>>
>> However, I think I did this at the same time as the last suggestion,
>> so I never saw the same error that you have.
>>
>> Jason
>>
>> On 6/7/11 4:45 PM, Gary Xie wrote:
>>
>> Thanks, Jason,
>>
>> Here are details on the error message I got.
>>
>> Gary
>>
>>
>>      formatted source viewer:
>>
>
/usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.d

> efault.user.config
>>
>>        [interface]
>>
>>      * classification = file / indexing
>>
>>
>>        [parameters]
>>
>>      * enter T for protein, F for nucleotide
>>      * $;PROTEIN$; = F
>>
>>
>>        [input]
>>
>>      * space-separated list
>>      * $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna
>>
>>
>>        [output]
>>
>>      * $;OUTPUT_TOKEN$; = default
>>      * $;OUTPUT_DIRECTORY$; =
>>
>
$;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_

> $;OUTPUT_TOKEN$;
>>      * call it whatever you like
>>      * $;DATABASE_NAME$; = nucdb
>>
>>
>>        [component]
>>
>>      * $;COMPONENT_NAME$; = formatdb
>>      * $;DESCRIPTION$; = none
>>      * $;WORKFLOW_REPOSITORY$; =
>>
>
$;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$
> ;OUTPUT_TOKEN$;
>>      * $;PIPELINE_TOKEN$; = unnamed
>>      * The version,revision,tag here is set by an interpolated CVS
tag

>>      * $;VERSION$; = 2.0
>>      * $;RELEASE_TAG$; = $Name$
>>      * $;REVISION$; = $Revision: 6918 $
>>      * $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml
>>
>>
>>        [include]
>>
>>      * $;PROJECT_CONFIG$; =
>>        /usr/local/projects/gary/workflow/project.config
>>
>
<http://seed.lanl.gov/ergatis/cgi/view_formatted_ini_source.cgi?file=/us

> r/local/projects/gary/workflow/project.config>
>> On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:
>>
>>
>>
>>
>>
>> Hi Gary,
>>
>> I had a similar problem with that part part of replace_config_keys.pl
>> (i.e. the add_keys subroutine). For me anyway, it turned out that the
>> version of Config::Inifiles that I had gives an error when you try to
> do
>> setval() on a value that doesn't already exist.
>>
>> So I changed the original sub from:
>> sub add_keys{
>> my($cfg,$section,@keys) = @_;
>> $logger->debug("Adding user defined keys: ".@keys)
>> if($logger->is_debug());
>> foreach my $kv (@keys){
>> my($key,$value) = split(/=/,$kv);
>> $logger->debug("Adding user defined key $key=$value in section
>> [$section]") if($logger->is_debug());
>> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
>> if(!$ret){
>> $logger->logdie("Couldn't add key $delimeter $key
>> $delimeter=$value to section [$section]");
>> }
>> }
>> }
>>
>> to:
>> sub add_keys{
>> my($cfg,$section,@keys) = @_;
>> $logger->debug("Adding user defined keys: ".@keys)
>> if($logger->is_debug());
>> foreach my $kv (@keys){
>> my($key,$value) = split(/=/,$kv);
>> $logger->debug("Adding user defined key $key=$value in section
>> [$section]") if($logger->is_debug());
>> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
>> if(!$ret){
>> $cfg->newval($section, $delimeter.$key.$delimeter,$value);
>> #$logger->logdie("Couldn't add key $delimeter $key
>> $delimeter=$value to section [$section]");
>> }
>> }
>> }
>>
>> Jason
>>
>> On 6/7/11 4:11 PM, Gary Xie wrote:
>>
>>
>>
>> We managed to get the java environment setup and get past the earlier
>>
>>      problem, but now when I create a formatdb pipeline and run it, I
> get
>>      this error.
>>
>>          more replace_config_keys.stderr
>>
>>      Died with 'Couldn't add key $; PIPELINEID $;=22 to section
>>
>>      [component]' at /opt/ergatis/bin/replace_config_keys.pl line 241
>>
>>      and if I look at the formatdb.default.user.config
>>
>>      it doesn't contain the key, so it can't replace it.
>>
>>      But, if I look at ergatis/docs/formatdb.config, it does have the
> keys
>>      (with no values):
>>
>>      ;the following keys are replaced at runtime by the invocation
> script
>>      $;COMPONENT_CONFIG$; =
>>
>>      $;COMPONENT_XML$; =
>>
>>      $;PIPELINE_XML$; =
>>
>>      $;PIPELINEID$; =
>>
>>      I would guess it uses the formatdb.config to create the config
> file
>>      for the pipeline, but don't understand why I'm losing the keys
and

>>
>>      getting this error.
>>
>>      Can anyone help?
>>
>>      Thanks,
>>
>>      Gary
>>
>>
>
------------------------------------------------------------------------

> ------
>>      EditLive Enterprise is the world's most technically advanced
> content
>>      authoring tool. Experience the power of Track Changes, Inline
> Image
>>      Editing and ensure content is compliant with Accessibility
> Checking.
>>      http://p.sf.net/sfu/ephox-dev2dev
>>
>>      _______________________________________________
>>
>>      Ergatis-users mailing list
>>
>>      [hidden email]
>>      <mailto:[hidden email]>
>>
>>      https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>>
>>
>>
>
------------------------------------------------------------------------

> ------
>> EditLive Enterprise is the world's most technically advanced content
>> authoring tool. Experience the power of Track Changes, Inline Image
>> Editing and ensure content is compliant with Accessibility Checking.
>> http://p.sf.net/sfu/ephox-dev2dev
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> <mailto:[hidden email]>
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>>
>
------------------------------------------------------------------------
> ------
>> EditLive Enterprise is the world's most technically advanced content
>> authoring tool. Experience the power of Track Changes, Inline Image
>> Editing and ensure content is compliant with Accessibility Checking.
>>
>
http://p.sf.net/sfu/ephox-dev2dev_______________________________________
> ________
>> Ergatis-users mailing list [hidden email]
>> <mailto:[hidden email]>
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>>
>
------------------------------------------------------------------------
>>
>
------------------------------------------------------------------------
> ------
>> EditLive Enterprise is the world's most technically advanced content
>> authoring tool. Experience the power of Track Changes, Inline Image
>> Editing and ensure content is compliant with Accessibility Checking.
>> http://p.sf.net/sfu/ephox-dev2dev
>>
>
------------------------------------------------------------------------
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>
------------------------------------------------------------------------

> ------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
>
------------------------------------------------------------------------
------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

Jason Kwan
Hi Karthik,

In the version I have there is no mention of $;INPUT_FILE_LIST$; in the
formatdb component, just $;INPUT_FILES$; (have a look at /<ergatis
dir>/docs/formatdb.config and /<ergatis dir>/docs/formatdb.xml), but I
just downloaded the new version of ergatis and there is an
$;INPUT_FILE_LIST$; in formatdb.config.  Which version do you have?  I'm
not sure if this is an intended change, but I suspect that there
shouldn't be any mention of $;INPUT_FILE_LIST$; in either
formatdb.config or formatdb.xml.

The input section of my version of formatdb.config looks like this:
[input]
;; space-separated list
$;INPUT_FILES$; =

And the actual second command part of my version of formatdb.xml looks
like this:
<command>
<type>RunUnixCommand</type>
<name>formatdb</name>
<state>incomplete</state>
<executable>$;FORMATDB_EXEC$;</executable>
<arg>-p $;PROTEIN$; -oT -i '$;INPUT_FILES$;' -n
$;OUTPUT_DIRECTORY$;/$;DATABASE_NAME$;</arg>
</command>

You could also try another component and see if this error only occurs
in the formatdb component.

Jason



On 6/14/11 1:46 PM, Muthuraman, Karthik Narayan (K) wrote:

> Thanks Jason. I had commented the lines out and hence the difference in
> line numbers. I had also made a mistake while copying the patch, which I
> fixed now.
>
> I get this error now:
> Died with 'Bad key $;INPUT_FILES$; in template file' at
> /apps/ergatis/bin/replace_template_keys.pl line 282
>
> In the interface, I gave the full path to my fasta file in "Input
> Files". But in workflow/runtime/pipeline/42/formatdb.default.config, the
> fasta file is assigned to " $;INPUT_FILE_LIST$;=/apps/ergatis/test.faa",
> and there is no $;INPUT_FILES$;
>
> Which file do I have to change ?
>
> Sincerely,
> Karthik Narayan
>
>
> -----Original Message-----
> From: Jason Kwan [mailto:[hidden email]]
> Sent: Tuesday, June 14, 2011 3:14 PM
> To: [hidden email]
> Subject: Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> The Dow Chemical Company's NEMX filtering software is automatically
> removing the following e-mail attachments from incoming messages as part
> of Dow's virus prevention program:
>
>
>
>       .bas, .bat, .bin, .chm, .cmd, .com, .cpl, .csc, .dll, .dbx, .exe,
> .hlp, .hta, .inf, .ins, .js, .jse,
>
>       .lnk, .msc, .msi, .msp, .mst, .nch, .pif .pl, .reg, .scr, .sct,
> .shb, .shs, .wsf, .wsh, .vb, .vbs
>       .vbe, .vxd, .wsc
>
>
> For clarification or more information, click here:
>
>
> http:////////////////manage-info.intranet.dow.com////////Information_Sec
> urity////////FAQ////////IS(FAQ)attachments.htmHi Karthik,
>
> I noticed that before you had the same error message but with line 241.
>
> Now it is line 269.  Now, in the version of replace_config_keys.pl that
> I have (suitable altered, as mentioned before), has only 260 lines.  I
> suspect it is one of two possibilities - 1. you have a different version
>
> of ergatis/the script (I have ergatis v2r13b2), or 2. maybe you added
> sections to the script rather than substituting?  I have attached the
> version that I have so you can compare.
>
> Thanks, and please let us know if this doesn't fix it.
>
> Jason
>
>
> On 6/14/11 12:52 PM, Muthuraman, Karthik Narayan (K) wrote:
>> Hello Aaron
>> I have set permissions to -R 777 for the workflow_manager and ergatis
>> directories for now. I don't believe it's a permission issue.
>>
>> When " /apps/ergatis/bin/replace_config_keys" is run, I get the error
>>
>> "Died with 'Couldn't add key $; PIPELINEID $;=33 to section
> [component]'
>> at ../ergatis/bin/replace_config_keys.pl line 269"
>>
>> Sincerely,
>> Karthik Narayan
>>
>>
>> -----Original Message-----
>> From: Aaron Buechlein [mailto:[hidden email]]
>> Sent: Tuesday, June 14, 2011 2:44 PM
>> To: [hidden email]
>> Subject: Re: [Ergatis-users] error message on the one-step workflow:
>> formatdb
>>
>> Hi Karthik,
>> This may be a silly question, but are you sure that the user running
>> apache has the same read/write permissions for the appropriate
>> directories as you do?
>> Thanks,
>>
>> Aaron
>>
>> Muthuraman, Karthik Narayan (K) wrote:
>>> Thanks Jason. Also, when I run the shell script "pipeline.32.run.sh"
>>> under workflows/scripts from my shell, it createdsthe log file,
> though
>>> with errors.
>>>
>>> But when ergatis runs it as apache, it does not create the log file
> at
>>> all.
>>>
>>> I have provided all the environment vars.
>>>
>>> Sincerely,
>>> Karthik Narayan
>>>
>>> *From:* Jason Kwan [mailto:[hidden email]]
>>> *Sent:* Tuesday, June 14, 2011 2:13 PM
>>> *To:* Muthuraman, Karthik Narayan (K)
>>> *Cc:* [hidden email]
>>> *Subject:* Re: [Ergatis-users] error message on the one-step
> workflow:
>>> formatdb
>>>
>>> Hi Karthik,
>>>
>>> You got past the "create output directory" and "create temp
> directory"
>>> stages so I can conclude that you probably have the right permissions
>>> for the output directory (do these get created)? Other than that
> there
>>> is not much to go on, apart from exactly the same error message that
>>> Gary faced. If I was faced with this situation, I would double check
>>> that I had changed replace_config_keys.pl in the install directory,
>>> rather than the directory that I had unpacked the ergatis source
>>> package (sudo find / -name "replace_config_keys.pl"), then change all
>>> of them just to be safe (and keep a backup of the original of
> course).
>>> Other than that, I can't think of anything. Perhaps the devs have
> some
>>> further thoughts?
>>>
>>> Jason
>>>
>>> On 6/14/11 11:42 AM, Muthuraman, Karthik Narayan (K) wrote:
>>>
>>> Hello Jason
>>>
>>> I looked at the thread, and patches proposed did not fix my problem.
>>>
>>> There are 2 params in the repace_config_keys.pl, that do not exist.
>>> They are:
>>>
>>>
> COMPONENT_XML=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_
>> default/component.xml
>>> --
>>>
> output_conf=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_de
>> fault/formatdb.default.final.config
>>> These two files are not created. Any thoughts ?
>>>
>>> Sincerely,
>>> Karthik Narayan
>>>
>>> *From:* Jason Kwan [mailto:[hidden email]]
>>> *Sent:* Tuesday, June 14, 2011 1:15 PM
>>> *To:* [hidden email]
>>> <mailto:[hidden email]>
>>> *Subject:* Re: [Ergatis-users] error message on the one-step
> workflow:
>>> formatdb
>>>
>>> Hi Karthik,
>>>
>>> This could be the bug that was described on this list on the 7th and
>>> 8th of June, please look back in the archives and let us know if it
>>> fixes it.
>>>
>>> Thanks
>>>
>>> Jason
>>>
>>>
>>> On 6/14/11 9:50 AM, Muthuraman, Karthik Narayan (K) wrote:
>>>
>>> Hello All
>>>
>>> Thanks for your responses.
>>>
>>> I have narrowed down the possible causes for the errors.
>>>
>>> When I run the shell script under workflows/scripts, it does not
>>> produce any output. But when I run the final command "RunWorkflow" in
>>> the script, it produces the log file. Meaning the environment
>>> variables created in the script are wrong. However when I open a new
>>> shell and try running the command again, it does not work. So, there
>>> is some environment variable that is working in the current shell I'm
>>> working on, and I don't know what that is. The only environment
>>> variable I added to the apache conf was PERL%LIB.
>>>
>>> Even when I ran the RunWorkflow command manually, I got this error :
>>>
>>> [root@nextgen ergatis]# cat
>>>
> projects/maize/workflow/runtime/formatdb/15_default/replace_config_keys.
>> stderr
>>> Died with 'Couldn't add key $; PIPELINEID $;=15 to section
>>> [component]' at ../ergatis/bin/replace_config_keys.pl line 241
>>>
>>> I'm just trying to run formatdb on a fasta file.
>>>
>>> Sincerely,
>>> Karthik Narayan
>>>
>>> *From:* Gary Xie [mailto:[hidden email]]
>>> *Sent:* Tuesday, June 14, 2011 11:04 AM
>>> *To:* Kevin Galens
>>> *Cc:* [hidden email]
>>> <mailto:[hidden email]>
>>> *Subject:* Re: [Ergatis-users] error message on the one-step
> workflow:
>>> formatdb
>>>
>>> Hi, all,
>>>
>>> We hit another glitch; It seems the code that runs the workflow is
>>> broken. I was hoping to wait for the new release. But we could post
>>> the issue here:
>>>
>>> It looks like whatever is creating the name of the file passed into
>>> the script that fails
>>>
>>> opt/ergatis/bin/create_file_iterator_list.pl<http://t.pl/>   line 140.
>>>
>>> is not substituting the pipleine id into the file name:
>>>
>>> the command is:
>>>
>>> /opt/ergatis/bin/create_file_iterator_list --input_directory=
>>> --input_directory_extension=fsa
>>> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
>>>
> --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblast
>> n/*_default*/i1.list
>>> --timestamp=0
>>>
>>> and should be
>>>
>>> /opt/ergatis/bin/create_file_iterator_list --input_directory=
>>> --input_directory_extension=fsa
>>> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
>>>
> --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblast
>> n/*10_default*/i1.list
>>> --timestamp=0
>>>
>>> I don't know where in the ergatis code this substitution should
>>> happen. Any insight?
>>>
>>> Thanks,
>>>
>>> Gary
>>>
>>> On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:
>>>
>>>
>>>
>>>
>>>
>>> Yes, this definitely is a bug. Just out of curiosity, what version of
>>> the module are you using? It would be great if you setup a ticket on
>>> Sourceforge so we can take a look at this issue. Include your
> proposed
>>> changes as well.
>>>
>>> http://sourceforge.net/tracker/?group_id=148765&atid=772583
>>> <http://sourceforge.net/tracker/?group_id=148765&atid=772583>
>>>
>>> Thank you for figuring this out.
>>>
>>> Kevin
>>>
>>> *From: *Jason Kwan<[hidden email]<mailto:[hidden email]>>
>>> *Organization: *University of Utah
>>> *Reply-To: *<[hidden email]<mailto:[hidden email]>>,
>>> <[hidden email]
>>> <mailto:[hidden email]>>
>>> *Date: *Tue, 07 Jun 2011 19:20:55 -0600
>>> *To: *Gary Xie<[hidden email]<mailto:[hidden email]>>
>>> *Cc: *"[hidden email]
>>> <mailto:[hidden email]>"
>>> <[hidden email]
>>> <mailto:[hidden email]>>
>>> *Subject: *Re: [Ergatis-users] error message on the one-step
> workflow:
>>> formatdb
>>>
>>> Hi Everyone,
>>>
>>> I wonder what the Devs think of the error that Gary and I
> encountered.
>>> I suspect maybe Config::Inifiles used to allow setval() on something
>>> that didn't previously exist, and if so this should count as a bug?
>>>
>>> Jason
>>>
>>> On 6/7/11 5:12 PM, Gary Xie wrote:
>>>
>>> Jason, Thanks a lot for your help and insight.
>>>
>>> Gary
>>>
>>> On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:
>>>
>>>
>>>
>>>
>>>
>>> Hi Gary,
>>>
>>> I'm not if this will be the answer to your problem, but I found in my
>>> notes that I did an additional thing. There is another section of
>>> replace_config_keys.pl at ~line 62 that looks like this:
>>>
>>> #Write the output location of this file as key $;COMPONENT_CONFIG$;
>>> my $ret =
>>>
> $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
>> ns{'output_conf'});
>>> if(!$ret){
>>> $logger->logdie("Couldn't add key
>>> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
>>> section [component]");
>>> }
>>> &check_parameters($cfg,'component');
>>>
>>> Which I changed to this:
>>> #Write the output location of this file as key $;COMPONENT_CONFIG$;
>>> my $ret =
>>>
> $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
>> ns{'output_conf'});
>>> if(!$ret){
>>>
> $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
>> ns{'output_conf'});
>>> #$logger->logdie("Couldn't add key
>>> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
>>> section [component]");
>>> }
>>> &check_parameters($cfg,'component');
>>>
>>> However, I think I did this at the same time as the last suggestion,
>>> so I never saw the same error that you have.
>>>
>>> Jason
>>>
>>> On 6/7/11 4:45 PM, Gary Xie wrote:
>>>
>>> Thanks, Jason,
>>>
>>> Here are details on the error message I got.
>>>
>>> Gary
>>>
>>>
>>>       formatted source viewer:
>>>
> /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.d
>> efault.user.config
>>>         [interface]
>>>
>>>       * classification = file / indexing
>>>
>>>
>>>         [parameters]
>>>
>>>       * enter T for protein, F for nucleotide
>>>       * $;PROTEIN$; = F
>>>
>>>
>>>         [input]
>>>
>>>       * space-separated list
>>>       * $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna
>>>
>>>
>>>         [output]
>>>
>>>       * $;OUTPUT_TOKEN$; = default
>>>       * $;OUTPUT_DIRECTORY$; =
>>>
> $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_
>> $;OUTPUT_TOKEN$;
>>>       * call it whatever you like
>>>       * $;DATABASE_NAME$; = nucdb
>>>
>>>
>>>         [component]
>>>
>>>       * $;COMPONENT_NAME$; = formatdb
>>>       * $;DESCRIPTION$; = none
>>>       * $;WORKFLOW_REPOSITORY$; =
>>>
> $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$
>> ;OUTPUT_TOKEN$;
>>>       * $;PIPELINE_TOKEN$; = unnamed
>>>       * The version,revision,tag here is set by an interpolated CVS
> tag
>>>       * $;VERSION$; = 2.0
>>>       * $;RELEASE_TAG$; = $Name$
>>>       * $;REVISION$; = $Revision: 6918 $
>>>       * $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml
>>>
>>>
>>>         [include]
>>>
>>>       * $;PROJECT_CONFIG$; =
>>>         /usr/local/projects/gary/workflow/project.config
>>>
> <http://seed.lanl.gov/ergatis/cgi/view_formatted_ini_source.cgi?file=/us
>> r/local/projects/gary/workflow/project.config>
>>> On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:
>>>
>>>
>>>
>>>
>>>
>>> Hi Gary,
>>>
>>> I had a similar problem with that part part of replace_config_keys.pl
>>> (i.e. the add_keys subroutine). For me anyway, it turned out that the
>>> version of Config::Inifiles that I had gives an error when you try to
>> do
>>> setval() on a value that doesn't already exist.
>>>
>>> So I changed the original sub from:
>>> sub add_keys{
>>> my($cfg,$section,@keys) = @_;
>>> $logger->debug("Adding user defined keys: ".@keys)
>>> if($logger->is_debug());
>>> foreach my $kv (@keys){
>>> my($key,$value) = split(/=/,$kv);
>>> $logger->debug("Adding user defined key $key=$value in section
>>> [$section]") if($logger->is_debug());
>>> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
>>> if(!$ret){
>>> $logger->logdie("Couldn't add key $delimeter $key
>>> $delimeter=$value to section [$section]");
>>> }
>>> }
>>> }
>>>
>>> to:
>>> sub add_keys{
>>> my($cfg,$section,@keys) = @_;
>>> $logger->debug("Adding user defined keys: ".@keys)
>>> if($logger->is_debug());
>>> foreach my $kv (@keys){
>>> my($key,$value) = split(/=/,$kv);
>>> $logger->debug("Adding user defined key $key=$value in section
>>> [$section]") if($logger->is_debug());
>>> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
>>> if(!$ret){
>>> $cfg->newval($section, $delimeter.$key.$delimeter,$value);
>>> #$logger->logdie("Couldn't add key $delimeter $key
>>> $delimeter=$value to section [$section]");
>>> }
>>> }
>>> }
>>>
>>> Jason
>>>
>>> On 6/7/11 4:11 PM, Gary Xie wrote:
>>>
>>>
>>>
>>> We managed to get the java environment setup and get past the earlier
>>>
>>>       problem, but now when I create a formatdb pipeline and run it, I
>> get
>>>       this error.
>>>
>>>           more replace_config_keys.stderr
>>>
>>>       Died with 'Couldn't add key $; PIPELINEID $;=22 to section
>>>
>>>       [component]' at /opt/ergatis/bin/replace_config_keys.pl line 241
>>>
>>>       and if I look at the formatdb.default.user.config
>>>
>>>       it doesn't contain the key, so it can't replace it.
>>>
>>>       But, if I look at ergatis/docs/formatdb.config, it does have the
>> keys
>>>       (with no values):
>>>
>>>       ;the following keys are replaced at runtime by the invocation
>> script
>>>       $;COMPONENT_CONFIG$; =
>>>
>>>       $;COMPONENT_XML$; =
>>>
>>>       $;PIPELINE_XML$; =
>>>
>>>       $;PIPELINEID$; =
>>>
>>>       I would guess it uses the formatdb.config to create the config
>> file
>>>       for the pipeline, but don't understand why I'm losing the keys
> and
>>>       getting this error.
>>>
>>>       Can anyone help?
>>>
>>>       Thanks,
>>>
>>>       Gary
>>>
>>>
> ------------------------------------------------------------------------
>> ------
>>>       EditLive Enterprise is the world's most technically advanced
>> content
>>>       authoring tool. Experience the power of Track Changes, Inline
>> Image
>>>       Editing and ensure content is compliant with Accessibility
>> Checking.
>>>       http://p.sf.net/sfu/ephox-dev2dev
>>>
>>>       _______________________________________________
>>>
>>>       Ergatis-users mailing list
>>>
>>>       [hidden email]
>>>       <mailto:[hidden email]>
>>>
>>>       https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>
>>>
>>>
>>>
> ------------------------------------------------------------------------
>> ------
>>> EditLive Enterprise is the world's most technically advanced content
>>> authoring tool. Experience the power of Track Changes, Inline Image
>>> Editing and ensure content is compliant with Accessibility Checking.
>>> http://p.sf.net/sfu/ephox-dev2dev
>>> _______________________________________________
>>> Ergatis-users mailing list
>>> [hidden email]
>>> <mailto:[hidden email]>
>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>
>>>
> ------------------------------------------------------------------------
>> ------
>>> EditLive Enterprise is the world's most technically advanced content
>>> authoring tool. Experience the power of Track Changes, Inline Image
>>> Editing and ensure content is compliant with Accessibility Checking.
>>>
> http://p.sf.net/sfu/ephox-dev2dev_______________________________________
>> ________
>>> Ergatis-users mailing list [hidden email]
>>> <mailto:[hidden email]>
>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>
>>>
> ------------------------------------------------------------------------
> ------------------------------------------------------------------------
>> ------
>>> EditLive Enterprise is the world's most technically advanced content
>>> authoring tool. Experience the power of Track Changes, Inline Image
>>> Editing and ensure content is compliant with Accessibility Checking.
>>> http://p.sf.net/sfu/ephox-dev2dev
>>>
> ------------------------------------------------------------------------
>>> _______________________________________________
>>> Ergatis-users mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>
> ------------------------------------------------------------------------
>> ------
>> EditLive Enterprise is the world's most technically advanced content
>> authoring tool. Experience the power of Track Changes, Inline Image
>> Editing and ensure content is compliant with Accessibility Checking.
>> http://p.sf.net/sfu/ephox-dev2dev
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>>
> ------------------------------------------------------------------------
> ------
>> EditLive Enterprise is the world's most technically advanced content
>> authoring tool. Experience the power of Track Changes, Inline Image
>> Editing and ensure content is compliant with Accessibility Checking.
>> http://p.sf.net/sfu/ephox-dev2dev
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

kmuthuraman
Hello
I removed the $;INPUT_FILE_LIST$; from the formatdb.xml and
formatdb.config, and it ran successfully

When I tried to run the ncbi-blastn component, I'm running into the same
problem.
Died with 'Bad key $;INPUT_FILE_LIST$; in template file' at
/apps/ergatis/bin/replace_template_keys.pl line 282

I removed $;INPUT_FILE_LIST$; part from all files in
ergatis/docs/ncbi-blast*, but still getting the error. But I don't see
the $;INPUT_FILE_LIST$;  variable in
runtime/pipeline/57/ncbi-blastn.default.config

What am I missing ?

Sincerely,
Karthik Narayan


-----Original Message-----
From: Jason Kwan [mailto:[hidden email]]
Sent: Tuesday, June 14, 2011 4:56 PM
To: Muthuraman, Karthik Narayan (K)
Cc: [hidden email]
Subject: Re: [Ergatis-users] error message on the one-step workflow:
formatdb

Hi Karthik,

In the version I have there is no mention of $;INPUT_FILE_LIST$; in the
formatdb component, just $;INPUT_FILES$; (have a look at /<ergatis
dir>/docs/formatdb.config and /<ergatis dir>/docs/formatdb.xml), but I
just downloaded the new version of ergatis and there is an
$;INPUT_FILE_LIST$; in formatdb.config.  Which version do you have?  I'm

not sure if this is an intended change, but I suspect that there
shouldn't be any mention of $;INPUT_FILE_LIST$; in either
formatdb.config or formatdb.xml.

The input section of my version of formatdb.config looks like this:
[input]
;; space-separated list
$;INPUT_FILES$; =

And the actual second command part of my version of formatdb.xml looks
like this:
<command>
<type>RunUnixCommand</type>
<name>formatdb</name>
<state>incomplete</state>
<executable>$;FORMATDB_EXEC$;</executable>
<arg>-p $;PROTEIN$; -oT -i '$;INPUT_FILES$;' -n
$;OUTPUT_DIRECTORY$;/$;DATABASE_NAME$;</arg>
</command>

You could also try another component and see if this error only occurs
in the formatdb component.

Jason



On 6/14/11 1:46 PM, Muthuraman, Karthik Narayan (K) wrote:
> Thanks Jason. I had commented the lines out and hence the difference
in
> line numbers. I had also made a mistake while copying the patch, which
I
> fixed now.
>
> I get this error now:
> Died with 'Bad key $;INPUT_FILES$; in template file' at
> /apps/ergatis/bin/replace_template_keys.pl line 282
>
> In the interface, I gave the full path to my fasta file in "Input
> Files". But in workflow/runtime/pipeline/42/formatdb.default.config,
the
> fasta file is assigned to "
$;INPUT_FILE_LIST$;=/apps/ergatis/test.faa",

> and there is no $;INPUT_FILES$;
>
> Which file do I have to change ?
>
> Sincerely,
> Karthik Narayan
>
>
> -----Original Message-----
> From: Jason Kwan [mailto:[hidden email]]
> Sent: Tuesday, June 14, 2011 3:14 PM
> To: [hidden email]
> Subject: Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> The Dow Chemical Company's NEMX filtering software is automatically
> removing the following e-mail attachments from incoming messages as
part
> of Dow's virus prevention program:
>
>
>
>       .bas, .bat, .bin, .chm, .cmd, .com, .cpl, .csc, .dll, .dbx,
.exe,

> .hlp, .hta, .inf, .ins, .js, .jse,
>
>       .lnk, .msc, .msi, .msp, .mst, .nch, .pif .pl, .reg, .scr, .sct,
> .shb, .shs, .wsf, .wsh, .vb, .vbs
>       .vbe, .vxd, .wsc
>
>
> For clarification or more information, click here:
>
>
>
http:////////////////manage-info.intranet.dow.com////////Information_Sec
> urity////////FAQ////////IS(FAQ)attachments.htmHi Karthik,
>
> I noticed that before you had the same error message but with line
241.
>
> Now it is line 269.  Now, in the version of replace_config_keys.pl
that
> I have (suitable altered, as mentioned before), has only 260 lines.  I
> suspect it is one of two possibilities - 1. you have a different
version

>
> of ergatis/the script (I have ergatis v2r13b2), or 2. maybe you added
> sections to the script rather than substituting?  I have attached the
> version that I have so you can compare.
>
> Thanks, and please let us know if this doesn't fix it.
>
> Jason
>
>
> On 6/14/11 12:52 PM, Muthuraman, Karthik Narayan (K) wrote:
>> Hello Aaron
>> I have set permissions to -R 777 for the workflow_manager and ergatis
>> directories for now. I don't believe it's a permission issue.
>>
>> When " /apps/ergatis/bin/replace_config_keys" is run, I get the error
>>
>> "Died with 'Couldn't add key $; PIPELINEID $;=33 to section
> [component]'
>> at ../ergatis/bin/replace_config_keys.pl line 269"
>>
>> Sincerely,
>> Karthik Narayan
>>
>>
>> -----Original Message-----
>> From: Aaron Buechlein [mailto:[hidden email]]
>> Sent: Tuesday, June 14, 2011 2:44 PM
>> To: [hidden email]
>> Subject: Re: [Ergatis-users] error message on the one-step workflow:
>> formatdb
>>
>> Hi Karthik,
>> This may be a silly question, but are you sure that the user running
>> apache has the same read/write permissions for the appropriate
>> directories as you do?
>> Thanks,
>>
>> Aaron
>>
>> Muthuraman, Karthik Narayan (K) wrote:
>>> Thanks Jason. Also, when I run the shell script "pipeline.32.run.sh"
>>> under workflows/scripts from my shell, it createdsthe log file,
> though
>>> with errors.
>>>
>>> But when ergatis runs it as apache, it does not create the log file
> at
>>> all.
>>>
>>> I have provided all the environment vars.
>>>
>>> Sincerely,
>>> Karthik Narayan
>>>
>>> *From:* Jason Kwan [mailto:[hidden email]]
>>> *Sent:* Tuesday, June 14, 2011 2:13 PM
>>> *To:* Muthuraman, Karthik Narayan (K)
>>> *Cc:* [hidden email]
>>> *Subject:* Re: [Ergatis-users] error message on the one-step
> workflow:
>>> formatdb
>>>
>>> Hi Karthik,
>>>
>>> You got past the "create output directory" and "create temp
> directory"
>>> stages so I can conclude that you probably have the right
permissions
>>> for the output directory (do these get created)? Other than that
> there
>>> is not much to go on, apart from exactly the same error message that
>>> Gary faced. If I was faced with this situation, I would double check
>>> that I had changed replace_config_keys.pl in the install directory,
>>> rather than the directory that I had unpacked the ergatis source
>>> package (sudo find / -name "replace_config_keys.pl"), then change
all

>>> of them just to be safe (and keep a backup of the original of
> course).
>>> Other than that, I can't think of anything. Perhaps the devs have
> some
>>> further thoughts?
>>>
>>> Jason
>>>
>>> On 6/14/11 11:42 AM, Muthuraman, Karthik Narayan (K) wrote:
>>>
>>> Hello Jason
>>>
>>> I looked at the thread, and patches proposed did not fix my problem.
>>>
>>> There are 2 params in the repace_config_keys.pl, that do not exist.
>>> They are:
>>>
>>>
>
COMPONENT_XML=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_
>> default/component.xml
>>> --
>>>
>
output_conf=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_de

>> fault/formatdb.default.final.config
>>> These two files are not created. Any thoughts ?
>>>
>>> Sincerely,
>>> Karthik Narayan
>>>
>>> *From:* Jason Kwan [mailto:[hidden email]]
>>> *Sent:* Tuesday, June 14, 2011 1:15 PM
>>> *To:* [hidden email]
>>> <mailto:[hidden email]>
>>> *Subject:* Re: [Ergatis-users] error message on the one-step
> workflow:
>>> formatdb
>>>
>>> Hi Karthik,
>>>
>>> This could be the bug that was described on this list on the 7th and
>>> 8th of June, please look back in the archives and let us know if it
>>> fixes it.
>>>
>>> Thanks
>>>
>>> Jason
>>>
>>>
>>> On 6/14/11 9:50 AM, Muthuraman, Karthik Narayan (K) wrote:
>>>
>>> Hello All
>>>
>>> Thanks for your responses.
>>>
>>> I have narrowed down the possible causes for the errors.
>>>
>>> When I run the shell script under workflows/scripts, it does not
>>> produce any output. But when I run the final command "RunWorkflow"
in
>>> the script, it produces the log file. Meaning the environment
>>> variables created in the script are wrong. However when I open a new
>>> shell and try running the command again, it does not work. So, there
>>> is some environment variable that is working in the current shell
I'm
>>> working on, and I don't know what that is. The only environment
>>> variable I added to the apache conf was PERL%LIB.
>>>
>>> Even when I ran the RunWorkflow command manually, I got this error :
>>>
>>> [root@nextgen ergatis]# cat
>>>
>
projects/maize/workflow/runtime/formatdb/15_default/replace_config_keys.

>> stderr
>>> Died with 'Couldn't add key $; PIPELINEID $;=15 to section
>>> [component]' at ../ergatis/bin/replace_config_keys.pl line 241
>>>
>>> I'm just trying to run formatdb on a fasta file.
>>>
>>> Sincerely,
>>> Karthik Narayan
>>>
>>> *From:* Gary Xie [mailto:[hidden email]]
>>> *Sent:* Tuesday, June 14, 2011 11:04 AM
>>> *To:* Kevin Galens
>>> *Cc:* [hidden email]
>>> <mailto:[hidden email]>
>>> *Subject:* Re: [Ergatis-users] error message on the one-step
> workflow:
>>> formatdb
>>>
>>> Hi, all,
>>>
>>> We hit another glitch; It seems the code that runs the workflow is
>>> broken. I was hoping to wait for the new release. But we could post
>>> the issue here:
>>>
>>> It looks like whatever is creating the name of the file passed into
>>> the script that fails
>>>
>>> opt/ergatis/bin/create_file_iterator_list.pl<http://t.pl/>   line
140.

>>>
>>> is not substituting the pipleine id into the file name:
>>>
>>> the command is:
>>>
>>> /opt/ergatis/bin/create_file_iterator_list --input_directory=
>>> --input_directory_extension=fsa
>>> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
>>>
>
--output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblast

>> n/*_default*/i1.list
>>> --timestamp=0
>>>
>>> and should be
>>>
>>> /opt/ergatis/bin/create_file_iterator_list --input_directory=
>>> --input_directory_extension=fsa
>>> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
>>>
>
--output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblast

>> n/*10_default*/i1.list
>>> --timestamp=0
>>>
>>> I don't know where in the ergatis code this substitution should
>>> happen. Any insight?
>>>
>>> Thanks,
>>>
>>> Gary
>>>
>>> On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:
>>>
>>>
>>>
>>>
>>>
>>> Yes, this definitely is a bug. Just out of curiosity, what version
of

>>> the module are you using? It would be great if you setup a ticket on
>>> Sourceforge so we can take a look at this issue. Include your
> proposed
>>> changes as well.
>>>
>>> http://sourceforge.net/tracker/?group_id=148765&atid=772583
>>> <http://sourceforge.net/tracker/?group_id=148765&atid=772583>
>>>
>>> Thank you for figuring this out.
>>>
>>> Kevin
>>>
>>> *From: *Jason Kwan<[hidden email]<mailto:[hidden email]>>
>>> *Organization: *University of Utah
>>> *Reply-To: *<[hidden email]<mailto:[hidden email]>>,
>>> <[hidden email]
>>> <mailto:[hidden email]>>
>>> *Date: *Tue, 07 Jun 2011 19:20:55 -0600
>>> *To: *Gary Xie<[hidden email]<mailto:[hidden email]>>
>>> *Cc: *"[hidden email]
>>> <mailto:[hidden email]>"
>>> <[hidden email]
>>> <mailto:[hidden email]>>
>>> *Subject: *Re: [Ergatis-users] error message on the one-step
> workflow:
>>> formatdb
>>>
>>> Hi Everyone,
>>>
>>> I wonder what the Devs think of the error that Gary and I
> encountered.
>>> I suspect maybe Config::Inifiles used to allow setval() on something
>>> that didn't previously exist, and if so this should count as a bug?
>>>
>>> Jason
>>>
>>> On 6/7/11 5:12 PM, Gary Xie wrote:
>>>
>>> Jason, Thanks a lot for your help and insight.
>>>
>>> Gary
>>>
>>> On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:
>>>
>>>
>>>
>>>
>>>
>>> Hi Gary,
>>>
>>> I'm not if this will be the answer to your problem, but I found in
my
>>> notes that I did an additional thing. There is another section of
>>> replace_config_keys.pl at ~line 62 that looks like this:
>>>
>>> #Write the output location of this file as key $;COMPONENT_CONFIG$;
>>> my $ret =
>>>
>
$cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio

>> ns{'output_conf'});
>>> if(!$ret){
>>> $logger->logdie("Couldn't add key
>>> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
>>> section [component]");
>>> }
>>> &check_parameters($cfg,'component');
>>>
>>> Which I changed to this:
>>> #Write the output location of this file as key $;COMPONENT_CONFIG$;
>>> my $ret =
>>>
>
$cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
>> ns{'output_conf'});
>>> if(!$ret){
>>>
>
$cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio

>> ns{'output_conf'});
>>> #$logger->logdie("Couldn't add key
>>> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
>>> section [component]");
>>> }
>>> &check_parameters($cfg,'component');
>>>
>>> However, I think I did this at the same time as the last suggestion,
>>> so I never saw the same error that you have.
>>>
>>> Jason
>>>
>>> On 6/7/11 4:45 PM, Gary Xie wrote:
>>>
>>> Thanks, Jason,
>>>
>>> Here are details on the error message I got.
>>>
>>> Gary
>>>
>>>
>>>       formatted source viewer:
>>>
>
/usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.d

>> efault.user.config
>>>         [interface]
>>>
>>>       * classification = file / indexing
>>>
>>>
>>>         [parameters]
>>>
>>>       * enter T for protein, F for nucleotide
>>>       * $;PROTEIN$; = F
>>>
>>>
>>>         [input]
>>>
>>>       * space-separated list
>>>       * $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna
>>>
>>>
>>>         [output]
>>>
>>>       * $;OUTPUT_TOKEN$; = default
>>>       * $;OUTPUT_DIRECTORY$; =
>>>
>
$;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_

>> $;OUTPUT_TOKEN$;
>>>       * call it whatever you like
>>>       * $;DATABASE_NAME$; = nucdb
>>>
>>>
>>>         [component]
>>>
>>>       * $;COMPONENT_NAME$; = formatdb
>>>       * $;DESCRIPTION$; = none
>>>       * $;WORKFLOW_REPOSITORY$; =
>>>
>
$;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$

>> ;OUTPUT_TOKEN$;
>>>       * $;PIPELINE_TOKEN$; = unnamed
>>>       * The version,revision,tag here is set by an interpolated CVS
> tag
>>>       * $;VERSION$; = 2.0
>>>       * $;RELEASE_TAG$; = $Name$
>>>       * $;REVISION$; = $Revision: 6918 $
>>>       * $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml
>>>
>>>
>>>         [include]
>>>
>>>       * $;PROJECT_CONFIG$; =
>>>         /usr/local/projects/gary/workflow/project.config
>>>
>
<http://seed.lanl.gov/ergatis/cgi/view_formatted_ini_source.cgi?file=/us

>> r/local/projects/gary/workflow/project.config>
>>> On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:
>>>
>>>
>>>
>>>
>>>
>>> Hi Gary,
>>>
>>> I had a similar problem with that part part of
replace_config_keys.pl
>>> (i.e. the add_keys subroutine). For me anyway, it turned out that
the
>>> version of Config::Inifiles that I had gives an error when you try
to

>> do
>>> setval() on a value that doesn't already exist.
>>>
>>> So I changed the original sub from:
>>> sub add_keys{
>>> my($cfg,$section,@keys) = @_;
>>> $logger->debug("Adding user defined keys: ".@keys)
>>> if($logger->is_debug());
>>> foreach my $kv (@keys){
>>> my($key,$value) = split(/=/,$kv);
>>> $logger->debug("Adding user defined key $key=$value in section
>>> [$section]") if($logger->is_debug());
>>> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
>>> if(!$ret){
>>> $logger->logdie("Couldn't add key $delimeter $key
>>> $delimeter=$value to section [$section]");
>>> }
>>> }
>>> }
>>>
>>> to:
>>> sub add_keys{
>>> my($cfg,$section,@keys) = @_;
>>> $logger->debug("Adding user defined keys: ".@keys)
>>> if($logger->is_debug());
>>> foreach my $kv (@keys){
>>> my($key,$value) = split(/=/,$kv);
>>> $logger->debug("Adding user defined key $key=$value in section
>>> [$section]") if($logger->is_debug());
>>> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
>>> if(!$ret){
>>> $cfg->newval($section, $delimeter.$key.$delimeter,$value);
>>> #$logger->logdie("Couldn't add key $delimeter $key
>>> $delimeter=$value to section [$section]");
>>> }
>>> }
>>> }
>>>
>>> Jason
>>>
>>> On 6/7/11 4:11 PM, Gary Xie wrote:
>>>
>>>
>>>
>>> We managed to get the java environment setup and get past the
earlier
>>>
>>>       problem, but now when I create a formatdb pipeline and run it,
I
>> get
>>>       this error.
>>>
>>>           more replace_config_keys.stderr
>>>
>>>       Died with 'Couldn't add key $; PIPELINEID $;=22 to section
>>>
>>>       [component]' at /opt/ergatis/bin/replace_config_keys.pl line
241
>>>
>>>       and if I look at the formatdb.default.user.config
>>>
>>>       it doesn't contain the key, so it can't replace it.
>>>
>>>       But, if I look at ergatis/docs/formatdb.config, it does have
the

>> keys
>>>       (with no values):
>>>
>>>       ;the following keys are replaced at runtime by the invocation
>> script
>>>       $;COMPONENT_CONFIG$; =
>>>
>>>       $;COMPONENT_XML$; =
>>>
>>>       $;PIPELINE_XML$; =
>>>
>>>       $;PIPELINEID$; =
>>>
>>>       I would guess it uses the formatdb.config to create the config
>> file
>>>       for the pipeline, but don't understand why I'm losing the keys
> and
>>>       getting this error.
>>>
>>>       Can anyone help?
>>>
>>>       Thanks,
>>>
>>>       Gary
>>>
>>>
>
------------------------------------------------------------------------

>> ------
>>>       EditLive Enterprise is the world's most technically advanced
>> content
>>>       authoring tool. Experience the power of Track Changes, Inline
>> Image
>>>       Editing and ensure content is compliant with Accessibility
>> Checking.
>>>       http://p.sf.net/sfu/ephox-dev2dev
>>>
>>>       _______________________________________________
>>>
>>>       Ergatis-users mailing list
>>>
>>>       [hidden email]
>>>       <mailto:[hidden email]>
>>>
>>>       https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>
>>>
>>>
>>>
>
------------------------------------------------------------------------

>> ------
>>> EditLive Enterprise is the world's most technically advanced content
>>> authoring tool. Experience the power of Track Changes, Inline Image
>>> Editing and ensure content is compliant with Accessibility Checking.
>>> http://p.sf.net/sfu/ephox-dev2dev
>>> _______________________________________________
>>> Ergatis-users mailing list
>>> [hidden email]
>>> <mailto:[hidden email]>
>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>
>>>
>
------------------------------------------------------------------------
>> ------
>>> EditLive Enterprise is the world's most technically advanced content
>>> authoring tool. Experience the power of Track Changes, Inline Image
>>> Editing and ensure content is compliant with Accessibility Checking.
>>>
>
http://p.sf.net/sfu/ephox-dev2dev_______________________________________
>> ________
>>> Ergatis-users mailing list [hidden email]
>>> <mailto:[hidden email]>
>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>
>>>
>
------------------------------------------------------------------------
>
------------------------------------------------------------------------
>> ------
>>> EditLive Enterprise is the world's most technically advanced content
>>> authoring tool. Experience the power of Track Changes, Inline Image
>>> Editing and ensure content is compliant with Accessibility Checking.
>>> http://p.sf.net/sfu/ephox-dev2dev
>>>
>
------------------------------------------------------------------------
>>> _______________________________________________
>>> Ergatis-users mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>
>
------------------------------------------------------------------------

>> ------
>> EditLive Enterprise is the world's most technically advanced content
>> authoring tool. Experience the power of Track Changes, Inline Image
>> Editing and ensure content is compliant with Accessibility Checking.
>> http://p.sf.net/sfu/ephox-dev2dev
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>>
>
------------------------------------------------------------------------
> ------
>> EditLive Enterprise is the world's most technically advanced content
>> authoring tool. Experience the power of Track Changes, Inline Image
>> Editing and ensure content is compliant with Accessibility Checking.
>> http://p.sf.net/sfu/ephox-dev2dev
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: error message on the one-step workflow: formatdb

Jason Kwan-2
Hi Karthik,

I did not suggest to make this change indescriminately, just in the formatdb component because it should not have $;INPUT_FILE_LIST$;.  Do the other components work in their original form?

Jason

Sent from my iPhone

On Jun 15, 2011, at 7:40 AM, "Muthuraman, Karthik Narayan (K)" <[hidden email]> wrote:

> Hello
> I removed the $;INPUT_FILE_LIST$; from the formatdb.xml and
> formatdb.config, and it ran successfully
>
> When I tried to run the ncbi-blastn component, I'm running into the same
> problem.
> Died with 'Bad key $;INPUT_FILE_LIST$; in template file' at
> /apps/ergatis/bin/replace_template_keys.pl line 282
>
> I removed $;INPUT_FILE_LIST$; part from all files in
> ergatis/docs/ncbi-blast*, but still getting the error. But I don't see
> the $;INPUT_FILE_LIST$;  variable in
> runtime/pipeline/57/ncbi-blastn.default.config
>
> What am I missing ?
>
> Sincerely,
> Karthik Narayan
>
>
> -----Original Message-----
> From: Jason Kwan [mailto:[hidden email]]
> Sent: Tuesday, June 14, 2011 4:56 PM
> To: Muthuraman, Karthik Narayan (K)
> Cc: [hidden email]
> Subject: Re: [Ergatis-users] error message on the one-step workflow:
> formatdb
>
> Hi Karthik,
>
> In the version I have there is no mention of $;INPUT_FILE_LIST$; in the
> formatdb component, just $;INPUT_FILES$; (have a look at /<ergatis
> dir>/docs/formatdb.config and /<ergatis dir>/docs/formatdb.xml), but I
> just downloaded the new version of ergatis and there is an
> $;INPUT_FILE_LIST$; in formatdb.config.  Which version do you have?  I'm
>
> not sure if this is an intended change, but I suspect that there
> shouldn't be any mention of $;INPUT_FILE_LIST$; in either
> formatdb.config or formatdb.xml.
>
> The input section of my version of formatdb.config looks like this:
> [input]
> ;; space-separated list
> $;INPUT_FILES$; =
>
> And the actual second command part of my version of formatdb.xml looks
> like this:
> <command>
> <type>RunUnixCommand</type>
> <name>formatdb</name>
> <state>incomplete</state>
> <executable>$;FORMATDB_EXEC$;</executable>
> <arg>-p $;PROTEIN$; -oT -i '$;INPUT_FILES$;' -n
> $;OUTPUT_DIRECTORY$;/$;DATABASE_NAME$;</arg>
> </command>
>
> You could also try another component and see if this error only occurs
> in the formatdb component.
>
> Jason
>
>
>
> On 6/14/11 1:46 PM, Muthuraman, Karthik Narayan (K) wrote:
>> Thanks Jason. I had commented the lines out and hence the difference
> in
>> line numbers. I had also made a mistake while copying the patch, which
> I
>> fixed now.
>>
>> I get this error now:
>> Died with 'Bad key $;INPUT_FILES$; in template file' at
>> /apps/ergatis/bin/replace_template_keys.pl line 282
>>
>> In the interface, I gave the full path to my fasta file in "Input
>> Files". But in workflow/runtime/pipeline/42/formatdb.default.config,
> the
>> fasta file is assigned to "
> $;INPUT_FILE_LIST$;=/apps/ergatis/test.faa",
>> and there is no $;INPUT_FILES$;
>>
>> Which file do I have to change ?
>>
>> Sincerely,
>> Karthik Narayan
>>
>>
>> -----Original Message-----
>> From: Jason Kwan [mailto:[hidden email]]
>> Sent: Tuesday, June 14, 2011 3:14 PM
>> To: [hidden email]
>> Subject: Re: [Ergatis-users] error message on the one-step workflow:
>> formatdb
>>
>> The Dow Chemical Company's NEMX filtering software is automatically
>> removing the following e-mail attachments from incoming messages as
> part
>> of Dow's virus prevention program:
>>
>>
>>
>>      .bas, .bat, .bin, .chm, .cmd, .com, .cpl, .csc, .dll, .dbx,
> .exe,
>> .hlp, .hta, .inf, .ins, .js, .jse,
>>
>>      .lnk, .msc, .msi, .msp, .mst, .nch, .pif .pl, .reg, .scr, .sct,
>> .shb, .shs, .wsf, .wsh, .vb, .vbs
>>      .vbe, .vxd, .wsc
>>
>>
>> For clarification or more information, click here:
>>
>>
>>
> http:////////////////manage-info.intranet.dow.com////////Information_Sec
>> urity////////FAQ////////IS(FAQ)attachments.htmHi Karthik,
>>
>> I noticed that before you had the same error message but with line
> 241.
>>
>> Now it is line 269.  Now, in the version of replace_config_keys.pl
> that
>> I have (suitable altered, as mentioned before), has only 260 lines.  I
>> suspect it is one of two possibilities - 1. you have a different
> version
>>
>> of ergatis/the script (I have ergatis v2r13b2), or 2. maybe you added
>> sections to the script rather than substituting?  I have attached the
>> version that I have so you can compare.
>>
>> Thanks, and please let us know if this doesn't fix it.
>>
>> Jason
>>
>>
>> On 6/14/11 12:52 PM, Muthuraman, Karthik Narayan (K) wrote:
>>> Hello Aaron
>>> I have set permissions to -R 777 for the workflow_manager and ergatis
>>> directories for now. I don't believe it's a permission issue.
>>>
>>> When " /apps/ergatis/bin/replace_config_keys" is run, I get the error
>>>
>>> "Died with 'Couldn't add key $; PIPELINEID $;=33 to section
>> [component]'
>>> at ../ergatis/bin/replace_config_keys.pl line 269"
>>>
>>> Sincerely,
>>> Karthik Narayan
>>>
>>>
>>> -----Original Message-----
>>> From: Aaron Buechlein [mailto:[hidden email]]
>>> Sent: Tuesday, June 14, 2011 2:44 PM
>>> To: [hidden email]
>>> Subject: Re: [Ergatis-users] error message on the one-step workflow:
>>> formatdb
>>>
>>> Hi Karthik,
>>> This may be a silly question, but are you sure that the user running
>>> apache has the same read/write permissions for the appropriate
>>> directories as you do?
>>> Thanks,
>>>
>>> Aaron
>>>
>>> Muthuraman, Karthik Narayan (K) wrote:
>>>> Thanks Jason. Also, when I run the shell script "pipeline.32.run.sh"
>>>> under workflows/scripts from my shell, it createdsthe log file,
>> though
>>>> with errors.
>>>>
>>>> But when ergatis runs it as apache, it does not create the log file
>> at
>>>> all.
>>>>
>>>> I have provided all the environment vars.
>>>>
>>>> Sincerely,
>>>> Karthik Narayan
>>>>
>>>> *From:* Jason Kwan [mailto:[hidden email]]
>>>> *Sent:* Tuesday, June 14, 2011 2:13 PM
>>>> *To:* Muthuraman, Karthik Narayan (K)
>>>> *Cc:* [hidden email]
>>>> *Subject:* Re: [Ergatis-users] error message on the one-step
>> workflow:
>>>> formatdb
>>>>
>>>> Hi Karthik,
>>>>
>>>> You got past the "create output directory" and "create temp
>> directory"
>>>> stages so I can conclude that you probably have the right
> permissions
>>>> for the output directory (do these get created)? Other than that
>> there
>>>> is not much to go on, apart from exactly the same error message that
>>>> Gary faced. If I was faced with this situation, I would double check
>>>> that I had changed replace_config_keys.pl in the install directory,
>>>> rather than the directory that I had unpacked the ergatis source
>>>> package (sudo find / -name "replace_config_keys.pl"), then change
> all
>>>> of them just to be safe (and keep a backup of the original of
>> course).
>>>> Other than that, I can't think of anything. Perhaps the devs have
>> some
>>>> further thoughts?
>>>>
>>>> Jason
>>>>
>>>> On 6/14/11 11:42 AM, Muthuraman, Karthik Narayan (K) wrote:
>>>>
>>>> Hello Jason
>>>>
>>>> I looked at the thread, and patches proposed did not fix my problem.
>>>>
>>>> There are 2 params in the repace_config_keys.pl, that do not exist.
>>>> They are:
>>>>
>>>>
>>
> COMPONENT_XML=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_
>>> default/component.xml
>>>> --
>>>>
>>
> output_conf=/apps/ergatis/projects/maize/workflow/runtime/formatdb/17_de
>>> fault/formatdb.default.final.config
>>>> These two files are not created. Any thoughts ?
>>>>
>>>> Sincerely,
>>>> Karthik Narayan
>>>>
>>>> *From:* Jason Kwan [mailto:[hidden email]]
>>>> *Sent:* Tuesday, June 14, 2011 1:15 PM
>>>> *To:* [hidden email]
>>>> <mailto:[hidden email]>
>>>> *Subject:* Re: [Ergatis-users] error message on the one-step
>> workflow:
>>>> formatdb
>>>>
>>>> Hi Karthik,
>>>>
>>>> This could be the bug that was described on this list on the 7th and
>>>> 8th of June, please look back in the archives and let us know if it
>>>> fixes it.
>>>>
>>>> Thanks
>>>>
>>>> Jason
>>>>
>>>>
>>>> On 6/14/11 9:50 AM, Muthuraman, Karthik Narayan (K) wrote:
>>>>
>>>> Hello All
>>>>
>>>> Thanks for your responses.
>>>>
>>>> I have narrowed down the possible causes for the errors.
>>>>
>>>> When I run the shell script under workflows/scripts, it does not
>>>> produce any output. But when I run the final command "RunWorkflow"
> in
>>>> the script, it produces the log file. Meaning the environment
>>>> variables created in the script are wrong. However when I open a new
>>>> shell and try running the command again, it does not work. So, there
>>>> is some environment variable that is working in the current shell
> I'm
>>>> working on, and I don't know what that is. The only environment
>>>> variable I added to the apache conf was PERL%LIB.
>>>>
>>>> Even when I ran the RunWorkflow command manually, I got this error :
>>>>
>>>> [root@nextgen ergatis]# cat
>>>>
>>
> projects/maize/workflow/runtime/formatdb/15_default/replace_config_keys.
>>> stderr
>>>> Died with 'Couldn't add key $; PIPELINEID $;=15 to section
>>>> [component]' at ../ergatis/bin/replace_config_keys.pl line 241
>>>>
>>>> I'm just trying to run formatdb on a fasta file.
>>>>
>>>> Sincerely,
>>>> Karthik Narayan
>>>>
>>>> *From:* Gary Xie [mailto:[hidden email]]
>>>> *Sent:* Tuesday, June 14, 2011 11:04 AM
>>>> *To:* Kevin Galens
>>>> *Cc:* [hidden email]
>>>> <mailto:[hidden email]>
>>>> *Subject:* Re: [Ergatis-users] error message on the one-step
>> workflow:
>>>> formatdb
>>>>
>>>> Hi, all,
>>>>
>>>> We hit another glitch; It seems the code that runs the workflow is
>>>> broken. I was hoping to wait for the new release. But we could post
>>>> the issue here:
>>>>
>>>> It looks like whatever is creating the name of the file passed into
>>>> the script that fails
>>>>
>>>> opt/ergatis/bin/create_file_iterator_list.pl<http://t.pl/>   line
> 140.
>>>>
>>>> is not substituting the pipleine id into the file name:
>>>>
>>>> the command is:
>>>>
>>>> /opt/ergatis/bin/create_file_iterator_list --input_directory=
>>>> --input_directory_extension=fsa
>>>> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
>>>>
>>
> --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblast
>>> n/*_default*/i1.list
>>>> --timestamp=0
>>>>
>>>> and should be
>>>>
>>>> /opt/ergatis/bin/create_file_iterator_list --input_directory=
>>>> --input_directory_extension=fsa
>>>> --input_file=/opt/ergatis/samples/proteins.faa --input_file_list=
>>>>
>>
> --output_iter_list=/usr/local/projects/gary/workflow/runtime/ncbi-tblast
>>> n/*10_default*/i1.list
>>>> --timestamp=0
>>>>
>>>> I don't know where in the ergatis code this substitution should
>>>> happen. Any insight?
>>>>
>>>> Thanks,
>>>>
>>>> Gary
>>>>
>>>> On Jun 8, 2011, at 7:12 AM, Kevin Galens wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Yes, this definitely is a bug. Just out of curiosity, what version
> of
>>>> the module are you using? It would be great if you setup a ticket on
>>>> Sourceforge so we can take a look at this issue. Include your
>> proposed
>>>> changes as well.
>>>>
>>>> http://sourceforge.net/tracker/?group_id=148765&atid=772583
>>>> <http://sourceforge.net/tracker/?group_id=148765&atid=772583>
>>>>
>>>> Thank you for figuring this out.
>>>>
>>>> Kevin
>>>>
>>>> *From: *Jason Kwan<[hidden email]<mailto:[hidden email]>>
>>>> *Organization: *University of Utah
>>>> *Reply-To: *<[hidden email]<mailto:[hidden email]>>,
>>>> <[hidden email]
>>>> <mailto:[hidden email]>>
>>>> *Date: *Tue, 07 Jun 2011 19:20:55 -0600
>>>> *To: *Gary Xie<[hidden email]<mailto:[hidden email]>>
>>>> *Cc: *"[hidden email]
>>>> <mailto:[hidden email]>"
>>>> <[hidden email]
>>>> <mailto:[hidden email]>>
>>>> *Subject: *Re: [Ergatis-users] error message on the one-step
>> workflow:
>>>> formatdb
>>>>
>>>> Hi Everyone,
>>>>
>>>> I wonder what the Devs think of the error that Gary and I
>> encountered.
>>>> I suspect maybe Config::Inifiles used to allow setval() on something
>>>> that didn't previously exist, and if so this should count as a bug?
>>>>
>>>> Jason
>>>>
>>>> On 6/7/11 5:12 PM, Gary Xie wrote:
>>>>
>>>> Jason, Thanks a lot for your help and insight.
>>>>
>>>> Gary
>>>>
>>>> On Jun 7, 2011, at 5:07 PM, Jason Kwan wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Hi Gary,
>>>>
>>>> I'm not if this will be the answer to your problem, but I found in
> my
>>>> notes that I did an additional thing. There is another section of
>>>> replace_config_keys.pl at ~line 62 that looks like this:
>>>>
>>>> #Write the output location of this file as key $;COMPONENT_CONFIG$;
>>>> my $ret =
>>>>
>>
> $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
>>> ns{'output_conf'});
>>>> if(!$ret){
>>>> $logger->logdie("Couldn't add key
>>>> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
>>>> section [component]");
>>>> }
>>>> &check_parameters($cfg,'component');
>>>>
>>>> Which I changed to this:
>>>> #Write the output location of this file as key $;COMPONENT_CONFIG$;
>>>> my $ret =
>>>>
>>
> $cfg->setval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
>>> ns{'output_conf'});
>>>> if(!$ret){
>>>>
>>
> $cfg->newval('component',$delimeter.'COMPONENT_CONFIG'.$delimeter,$optio
>>> ns{'output_conf'});
>>>> #$logger->logdie("Couldn't add key
>>>> ${delimeter}COMPONENT_CONFIG$delimeter=$options{'output_conf'}to
>>>> section [component]");
>>>> }
>>>> &check_parameters($cfg,'component');
>>>>
>>>> However, I think I did this at the same time as the last suggestion,
>>>> so I never saw the same error that you have.
>>>>
>>>> Jason
>>>>
>>>> On 6/7/11 4:45 PM, Gary Xie wrote:
>>>>
>>>> Thanks, Jason,
>>>>
>>>> Here are details on the error message I got.
>>>>
>>>> Gary
>>>>
>>>>
>>>>      formatted source viewer:
>>>>
>>
> /usr/local/projects/gary/workflow/runtime/formatdb/16_default/formatdb.d
>>> efault.user.config
>>>>        [interface]
>>>>
>>>>      * classification = file / indexing
>>>>
>>>>
>>>>        [parameters]
>>>>
>>>>      * enter T for protein, F for nucleotide
>>>>      * $;PROTEIN$; = F
>>>>
>>>>
>>>>        [input]
>>>>
>>>>      * space-separated list
>>>>      * $;INPUT_FILES$; = /opt/ergatis/samples/NC_010364.fna
>>>>
>>>>
>>>>        [output]
>>>>
>>>>      * $;OUTPUT_TOKEN$; = default
>>>>      * $;OUTPUT_DIRECTORY$; =
>>>>
>>
> $;REPOSITORY_ROOT$;/output_repository/$;COMPONENT_NAME$;/$;PIPELINEID$;_
>>> $;OUTPUT_TOKEN$;
>>>>      * call it whatever you like
>>>>      * $;DATABASE_NAME$; = nucdb
>>>>
>>>>
>>>>        [component]
>>>>
>>>>      * $;COMPONENT_NAME$; = formatdb
>>>>      * $;DESCRIPTION$; = none
>>>>      * $;WORKFLOW_REPOSITORY$; =
>>>>
>>
> $;REPOSITORY_ROOT$;/workflow/runtime/$;COMPONENT_NAME$;/$;PIPELINEID$;_$
>>> ;OUTPUT_TOKEN$;
>>>>      * $;PIPELINE_TOKEN$; = unnamed
>>>>      * The version,revision,tag here is set by an interpolated CVS
>> tag
>>>>      * $;VERSION$; = 2.0
>>>>      * $;RELEASE_TAG$; = $Name$
>>>>      * $;REVISION$; = $Revision: 6918 $
>>>>      * $;TEMPLATE_XML$; = $;DOCS_DIR$;/$;COMPONENT_NAME$;.xml
>>>>
>>>>
>>>>        [include]
>>>>
>>>>      * $;PROJECT_CONFIG$; =
>>>>        /usr/local/projects/gary/workflow/project.config
>>>>
>>
> <http://seed.lanl.gov/ergatis/cgi/view_formatted_ini_source.cgi?file=/us
>>> r/local/projects/gary/workflow/project.config>
>>>> On Jun 7, 2011, at 4:28 PM, Jason Kwan wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Hi Gary,
>>>>
>>>> I had a similar problem with that part part of
> replace_config_keys.pl
>>>> (i.e. the add_keys subroutine). For me anyway, it turned out that
> the
>>>> version of Config::Inifiles that I had gives an error when you try
> to
>>> do
>>>> setval() on a value that doesn't already exist.
>>>>
>>>> So I changed the original sub from:
>>>> sub add_keys{
>>>> my($cfg,$section,@keys) = @_;
>>>> $logger->debug("Adding user defined keys: ".@keys)
>>>> if($logger->is_debug());
>>>> foreach my $kv (@keys){
>>>> my($key,$value) = split(/=/,$kv);
>>>> $logger->debug("Adding user defined key $key=$value in section
>>>> [$section]") if($logger->is_debug());
>>>> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
>>>> if(!$ret){
>>>> $logger->logdie("Couldn't add key $delimeter $key
>>>> $delimeter=$value to section [$section]");
>>>> }
>>>> }
>>>> }
>>>>
>>>> to:
>>>> sub add_keys{
>>>> my($cfg,$section,@keys) = @_;
>>>> $logger->debug("Adding user defined keys: ".@keys)
>>>> if($logger->is_debug());
>>>> foreach my $kv (@keys){
>>>> my($key,$value) = split(/=/,$kv);
>>>> $logger->debug("Adding user defined key $key=$value in section
>>>> [$section]") if($logger->is_debug());
>>>> my $ret = $cfg->setval($section,$delimeter.$key.$delimeter,$value);
>>>> if(!$ret){
>>>> $cfg->newval($section, $delimeter.$key.$delimeter,$value);
>>>> #$logger->logdie("Couldn't add key $delimeter $key
>>>> $delimeter=$value to section [$section]");
>>>> }
>>>> }
>>>> }
>>>>
>>>> Jason
>>>>
>>>> On 6/7/11 4:11 PM, Gary Xie wrote:
>>>>
>>>>
>>>>
>>>> We managed to get the java environment setup and get past the
> earlier
>>>>
>>>>      problem, but now when I create a formatdb pipeline and run it,
> I
>>> get
>>>>      this error.
>>>>
>>>>          more replace_config_keys.stderr
>>>>
>>>>      Died with 'Couldn't add key $; PIPELINEID $;=22 to section
>>>>
>>>>      [component]' at /opt/ergatis/bin/replace_config_keys.pl line
> 241
>>>>
>>>>      and if I look at the formatdb.default.user.config
>>>>
>>>>      it doesn't contain the key, so it can't replace it.
>>>>
>>>>      But, if I look at ergatis/docs/formatdb.config, it does have
> the
>>> keys
>>>>      (with no values):
>>>>
>>>>      ;the following keys are replaced at runtime by the invocation
>>> script
>>>>      $;COMPONENT_CONFIG$; =
>>>>
>>>>      $;COMPONENT_XML$; =
>>>>
>>>>      $;PIPELINE_XML$; =
>>>>
>>>>      $;PIPELINEID$; =
>>>>
>>>>      I would guess it uses the formatdb.config to create the config
>>> file
>>>>      for the pipeline, but don't understand why I'm losing the keys
>> and
>>>>      getting this error.
>>>>
>>>>      Can anyone help?
>>>>
>>>>      Thanks,
>>>>
>>>>      Gary
>>>>
>>>>
>>
> ------------------------------------------------------------------------
>>> ------
>>>>      EditLive Enterprise is the world's most technically advanced
>>> content
>>>>      authoring tool. Experience the power of Track Changes, Inline
>>> Image
>>>>      Editing and ensure content is compliant with Accessibility
>>> Checking.
>>>>      http://p.sf.net/sfu/ephox-dev2dev
>>>>
>>>>      _______________________________________________
>>>>
>>>>      Ergatis-users mailing list
>>>>
>>>>      [hidden email]
>>>>      <mailto:[hidden email]>
>>>>
>>>>      https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>>
>>>>
>>>>
>>>>
>>
> ------------------------------------------------------------------------
>>> ------
>>>> EditLive Enterprise is the world's most technically advanced content
>>>> authoring tool. Experience the power of Track Changes, Inline Image
>>>> Editing and ensure content is compliant with Accessibility Checking.
>>>> http://p.sf.net/sfu/ephox-dev2dev
>>>> _______________________________________________
>>>> Ergatis-users mailing list
>>>> [hidden email]
>>>> <mailto:[hidden email]>
>>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>>
>>>>
>>
> ------------------------------------------------------------------------
>>> ------
>>>> EditLive Enterprise is the world's most technically advanced content
>>>> authoring tool. Experience the power of Track Changes, Inline Image
>>>> Editing and ensure content is compliant with Accessibility Checking.
>>>>
>>
> http://p.sf.net/sfu/ephox-dev2dev_______________________________________
>>> ________
>>>> Ergatis-users mailing list [hidden email]
>>>> <mailto:[hidden email]>
>>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>>
>>>>
>>
> ------------------------------------------------------------------------
>>
> ------------------------------------------------------------------------
>>> ------
>>>> EditLive Enterprise is the world's most technically advanced content
>>>> authoring tool. Experience the power of Track Changes, Inline Image
>>>> Editing and ensure content is compliant with Accessibility Checking.
>>>> http://p.sf.net/sfu/ephox-dev2dev
>>>>
>>
> ------------------------------------------------------------------------
>>>> _______________________________________________
>>>> Ergatis-users mailing list
>>>> [hidden email]
>>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>>
>>
> ------------------------------------------------------------------------
>>> ------
>>> EditLive Enterprise is the world's most technically advanced content
>>> authoring tool. Experience the power of Track Changes, Inline Image
>>> Editing and ensure content is compliant with Accessibility Checking.
>>> http://p.sf.net/sfu/ephox-dev2dev
>>> _______________________________________________
>>> Ergatis-users mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>
>>>
>>
> ------------------------------------------------------------------------
>> ------
>>> EditLive Enterprise is the world's most technically advanced content
>>> authoring tool. Experience the power of Track Changes, Inline Image
>>> Editing and ensure content is compliant with Accessibility Checking.
>>> http://p.sf.net/sfu/ephox-dev2dev
>>> _______________________________________________
>>> Ergatis-users mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
12