Viewing 12 reply threads
  • Author
    Posts
    • #13593
      Glen Eustace
      Participant

        I have just installed OTOBO 10.1 using the docker instructions as my standalone deployment has been having issues with parsing text/quoted-plain email messages which I have been unable to resolve. The only deviation from the instructions was to not run the db container but instead to connect to the existing database.  I achieved this by using a socat relay on the docker host.

        Issue 1.

        Fetchmail is not being run, I am having to run it manually from the Admin>Postmaster Mail Accounts.

        Issue 2.

        Daemon container cant connect to the ElasticSearch, I have tried including the error message but WordFence keeps blocking it, I am happy to email instead but the message says that there is an error 500 whilst attempting to do a post to localhost:9200 to the controller /ticket/_delete_by_query

        ERROR: OTOBO-otobo.Daemon.pl – Daemon Kernel::System::Daemon::DaemonModules::SchedulerTaskWorker-10
        Perl: 5.36.0 OS: linux Time: Tue Aug 23 08:20:11 2022

        Traceback (12077):
        Module: Kernel::GenericInterface::ErrorHandling::HandleError Line: 115
        Module: Kernel::GenericInterface::Requester::_HandleError Line: 543
        Module: Kernel::GenericInterface::Requester::Run Line: 340
        Module: Kernel::GenericInterface::Event::Handler::Run Line: 214
        Module: (eval) Line: 232
        Module: Kernel::System::EventHandler::EventHandler Line: 226
        Module: Kernel::System::EventHandler::EventHandlerTransaction Line: 279
        Module: Kernel::System::ObjectManager::ObjectEventsHandle Line: 441
        Module: Kernel::System::Daemon::DaemonModules::SchedulerTaskWorker::Run Line: 246
        Module: (eval) Line: 386
        Module: main::Start Line: 386
        Module: bin/otobo.Daemon.pl Line: 164

      • #13594
        Glen Eustace
        Participant

          This is weird. If I run the Console command, it says there are no configured mail accounts, but if I use the web UI, and click on ‘Fetch Mail’, it works. If I look in the db directly, there is an entry in the mail_accounts table.

          otobo@997c99e8bd0b:~/Kernel$ ~/bin/otobo.Console.pl Maint::PostMaster::MailAccountFetch

          No configured mail accounts found!

           

        • #13598
          bes
          Participant

            Hi Glen,

            I have no idea about the mail configuration. But I noticed something in the Elasticsearch error message. The message says that the Daemon wants to talk to localhost:9200 . But in a regular Docker-based setup ES is running on elastic:9200. This ES configuration can be adapted in the Admin-Interface for “Webservices”.

            In new installations this should not happen as the initial insert into the database is changed in the Docker image. See https://github.com/RotherOSS/otobo/blob/rel-10_1/otobo.web.dockerfile#L114 . Did you migrate data from your old installation into the new installation?

            Best regards,

            Bernhard

             

          • #13602
            Glen Eustace
            Participant

              This is the regular docker install, but I guess that I might have overwritten the config as I used my existing database.

              I didn’t migrate the data, I simple attached the database by updating the Kernel>Config.pm

              You are 100% correct, the value for the container is wrong because that was how I had it in my original deployment.  I am struggling to work out how to fix it as the field in the database is a blob. I cant find where in the System Config I can fix this.

            • #13603
              Glen Eustace
              Participant

                OK, I think I worked that one out phpMyAdmin allows me to edit the blob by uploading a file.

              • #13604
                Glen Eustace
                Participant

                  Hmmm, that didn’t work.  It still is using localhost. I have tried rebuilding and sync’ing the config and stop and restarting the daemon.  No luck so far.

                • #13605
                  Glen Eustace
                  Participant

                    Well I don’t know what I have done but the ElasticSearch is now working, I am using the console.pl script Maint::Elasticsearch::Migration to load the history. 30% of tickets so far but looks good.

                    Running Maint::PostMaster::MailAccountFetch also now works. But I haven’t done anything that I am aware of that might have had an impact there.

                    BUT, the web interface no longer works at all.  I cant seem to find any error logs anywhere that give a clue as to what is happening. The URL just times out

                  • #13606
                    Glen Eustace
                    Participant

                      Solved, our fail2ban service had put my IP address into our boundary firewall.  Haven’t worked out why yet but at least I have access again. So, mail working again, elasticsearch not generating errors – progress.  At one point I had an input field at the top of the agent screen labelled elastic search.  This has also vanished so I am now on the hist to try to work out how to get that back again.

                    • #13608
                      Glen Eustace
                      Participant

                        Got that one nailed now too. Now hopefully with a functional system will see if has cured the issue with un-parsable emails.

                      • #13610
                        bes
                        Participant

                          Hi Glen,

                          glad to hear that your installation  is now in a workable state.

                          Using a database, which is not running under Docker, from a Docker-based web server is a valid use case. That the Elasticsearch Webservice Konfiguration must then be adapted manually is understandable. It’s hard to automate that, as many different setups are possible. But there shouldn’t have been a need to update the webservice configuration in the database directly. This should have been possible from the Admin-Interface.

                          I’m also curious about the socat relay. Was that necessary because the database can only be reached via an SSH-tunnel? My understanding was that the services running in Docker container can access the same network resources as are reachable from the Docker host.

                          Also, did you edit _docker-compose/otobo-base.yml_ in order to deactivate the Docker-based database service? I had been thinking about moving the service db into a dedicated configuration file. This would all to suppress the service by only editing the _.env_ file.

                          Best regards,

                          Bernhard

                           

                        • #13629
                          Glen Eustace
                          Participant

                            Finally getting to the bottom of the Email Parsing issue.  There seems to be an issue with MariaDB and collation.  I get the following error;

                            DBD::mysql::db do failed: Incorrect string value: ‘\xF0\x9F\x98\x8A\x0A\x0A…’ for column otobo.article_data_mime.a_body at row 1 at /opt/otobo/Kernel/System/DB.pm line 556.
                            ERROR: OTOBO-otobo.Console.pl-Maint::PostMaster::Read-10 Perl: 5.36.0 OS: linux Time: Mon Aug 29 23:21:07 2022

                            Message: Incorrect string value: ‘\xF0\x9F\x98\x8A\x0A\x0A…’ for column otobo.article_data_mime.a_body at row 1, SQL: ‘
                            INSERT INTO article_data_mime (
                            article_id, a_from, a_reply_to, a_to, a_cc, a_bcc, a_subject, a_message_id,
                            a_message_id_md5, a_in_reply_to, a_references, a_content_type, a_body,
                            incoming_time, content_path, create_time, create_by, change_time, change_by)
                            VALUES (
                            ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ‘2022-08-29 23:21:07’, ?, ‘2022-08-29 23:21:07’, ?
                            )

                            But the collation appears to be correct – it is currently utf8-general-ci, maybe its supposed to be mtf8mb4

                          • #13630
                            Glen Eustace
                            Participant

                              Yep, finally got there.  The DB and all the tables need to be utf8mb4 and utf8mb4-general-ci.  So converting to the docker version was probably a waste of time but did have the benefit of learning more stuff.

                               

                            • #13634
                              bes
                              Participant

                                Yes, the broken utf8 character set of MySQL is really annoying. It does not allow to store characters that are encoded by four octets. An example for such an character is a simple smiley:

                                bes:~ $ uni -8 1F601
                                😁 - U+1F601 - F0 9F 98 81 - GRINNING FACE WITH SMILING EYES

                                The fix is to check the character set of the OTOBO tables and of the columns of the tables. The character set must be utf8mb4. The mb4 stands for four bytes. See also https://github.com/RotherOSS/otobo/issues/1879 .

                                 

                            Viewing 12 reply threads
                            • You must be logged in to reply to this topic.