1790 – Guidance utilities cannot work in TDE R14

By default, Bugzilla does not search the list of RESOLVED bugs.
You can force it to do so by putting the upper-case word ALL in front of your search query, e.g.: ALL tdelibs
We recommend searching for bugs this way, as you may discover that your bug has already been resolved and fixed in a later release.

Bug 1790 - Guidance utilities cannot work in TDE R14

Summary: Guidance utilities cannot work in TDE R14

Status:	RESOLVED FIXED

Alias:	None

Product:	TDE
Classification:	Unclassified
Component:	non-core programs (show other bugs)
Version:	R14.0.0 [Trinity]
Hardware:	All Linux

Importance:	P5 normal
Assignee:	Francois Andriot

URL:

Depends on:
Blocks:

Reported:	2013-12-28 08:08 CST by Francois Andriot
Modified:	2014-02-08 15:16 CST (History)
CC List:	5 users (show)

See Also:
Compiler Version:
TDE Version String:
Application Version:
Application Name:

Attachments
python-tqt : fix detection of sip4-tqt (994 bytes, patch) 2013-12-29 03:39 CST, Francois Andriot	Details \| Diff
python-trinity : fix detection of sip4-tqt and python-tqt (951 bytes, patch) 2013-12-29 03:48 CST, Francois Andriot	Details \| Diff
pytdeextensions: convert QT to TQT in 'kdedistutils.py' (2.26 KB, patch) 2013-12-29 04:25 CST, Francois Andriot	Details \| Diff
pytdeextensions: add sip4-tqt and python-tqt to the Python search path (1.41 KB, patch) 2013-12-29 04:31 CST, Francois Andriot	Details \| Diff
pytdeextensions: adds the EXTRA_MODULES_DIR variable (1.37 KB, patch) 2013-12-29 04:39 CST, Francois Andriot	Details \| Diff
pytdeextensions: remove hardoded "lib" directory (1.21 KB, patch) 2013-12-29 04:39 CST, Francois Andriot	Details \| Diff
pytdeextensions: fix include directory location (787 bytes, patch) 2013-12-29 04:42 CST, Francois Andriot	Details \| Diff
pytdeextensions: sample resulting 'kdedistutils.py' file (95.63 KB, text/x-python) 2013-12-29 04:50 CST, Francois Andriot	Details
tde-guidance: fix 'userconfig' for TDE R14 (40.72 KB, patch) 2013-12-29 04:58 CST, Francois Andriot	Details \| Diff
python-trinity : rename kdepyuic => tdepyuic (10.49 KB, patch) 2014-01-12 12:50 CST, Slávek Banko	Details \| Diff
pytdeextensions: add sip4-tqt and python-tqt to the Python search path (1) (5.47 KB, patch) 2014-01-12 13:02 CST, Slávek Banko	Details \| Diff
pytdeextensions: remove hardoded "lib" directory (1) (3.23 KB, patch) 2014-01-12 13:11 CST, Slávek Banko	Details \| Diff
pytdeextensions: fix include directory location (1) (1.53 KB, patch) 2014-01-12 13:14 CST, Slávek Banko	Details \| Diff
pytdeextensions: rename kdedistutils => tdedistutils (201.34 KB, patch) 2014-01-12 13:39 CST, Slávek Banko	Details \| Diff
pytdeextensions: rename kdedesigner => tdedesigner (17.00 KB, patch) 2014-01-12 13:40 CST, Slávek Banko	Details \| Diff
pytdeextensions: rename kdedistutils => tdedistutils (1) (201.34 KB, patch) 2014-01-15 15:43 CST, Slávek Banko	Details \| Diff
tde-guidance: fix 'userconfig' for TDE R14 (1) (40.92 KB, patch) 2014-01-15 15:54 CST, Slávek Banko	Details \| Diff
tde-guidance: rename kdedistutils => tdedistutils (573 bytes, patch) 2014-01-15 15:58 CST, Slávek Banko	Details \| Diff
sip4-tqt : install as module by default (849 bytes, patch) 2014-02-04 18:03 CST, Slávek Banko	Details \| Diff
python-tqt : install as module by default (1007 bytes, patch) 2014-02-04 18:04 CST, Slávek Banko	Details \| Diff
Show Obsolete (19) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.

Description Francois Andriot 2013-12-28 08:08:24 CST

Hello, I'm currently in the process of making the Guidance utilities work in TDE R14. In 3.5.13.2 it was possible with patches (See bug http://bugs.pearsoncomputing.net/bugzilla/show_bug.cgi?id=1188 )

But in R14, it looks like the code has not even been converted from QT to TQT.

I will post here the various patch I need to make the stuff work.

Here are the utilities from Guidance:
- displayconfig: was deprecated in 3.5.13 => Won't fix
- powermanager: relies on HAL, is redundant with TDEpowersave => Won't fix
- userconfig: works in R14 => Will be ported.
- serviceconfig: only support legacy SYSV services. We should enhance it to support Systemd. => Will be ported.
- mountconfig: relies on HAL, but may be useful if we can patch it. => Won't fix now.
- wineconfig: apparently should be ported, but I do not know if it works with recent versions of HAL.

Comment 1 Francois Andriot 2013-12-28 08:15:47 CST

A word about SIP: since we have renamed QT3 to TQT3, we cannot use anymore the official SIP utilities to generate our python utilities. The official SIP only recognizes the "Q" variables and functions, not the "TQ".
We now MUST use our own SIP utility from our GIT. It is based on version 4.10.5 and works well for TDE.

BUT our SIP utility should co-exist with the distribution-provided one (used by KDE4, among other). So I chose to install the utilities under "/opt/trinity" and the Python modules under "/usr/lib64/pythonXX/site-packages/sip4-tqt" . The original SIP goes directly under "/usr/lib64/pythonXX/site-packages" so we need to be in a subdirectory.

Here are the build commands (change path according to your distribution):
mkdir build
cd build
python ../configure.py -b /opt/trinity/bin -d /usr/lib64/python2.6/site-packages/sip4-tqt -e /opt/trinity/include

Then I get the files:
/usr/lib64/python2.6/site-packages/sip4-tqt/sip.so
/usr/lib64/python2.6/site-packages/sip4-tqt/sipconfig.py
/usr/lib64/python2.6/site-packages/sip4-tqt/sipdistutils.py

And in development package:
/opt/trinity/bin/sip
/opt/trinity/include/sip.h

BTW, having our own "SIP" utility avoids the problem I currently have in openSUSE 13.1: python-trinity package FTBFS with upstream SIP 4.15 ...

Comment 2 Francois Andriot 2013-12-29 03:09:32 CST

(En réponse au commentaire 1)

Further advance in SIP compilation:
In order to properly use SIP as a python module (e.g. our scripts located in a subfolder of the python module directory), it needs to have an "__init__.py" script, which can be empty text file. Also, a python module cannot contain dash caracter, so I now use underscore.

Resulting files:
/usr/lib64/python2.7/site-packages/sip4_tqt/__init__.py
/usr/lib64/python2.7/site-packages/sip4_tqt/sip.so
/usr/lib64/python2.7/site-packages/sip4_tqt/sipconfig.py
/usr/lib64/python2.7/site-packages/sip4_tqt/sipdistutils.py

Development files:
/opt/trinity/bin/sip
/opt/trinity/include/sip.h

Please note that the "/usr/lib64/python2.7/site-packages" is the "Python Library directory", where the Python architecture-dependant modules are installed in a RPM distribution. It is given by the command:

python -c "from distutils.sysconfig import get_python_lib; print get_python_lib(1)"

Comment 3 Francois Andriot 2013-12-29 03:39:50 CST

Created attachment 1787 [details]
python-tqt : fix detection of sip4-tqt

The "python-tqt" package should now be installed to "<python_modules_dir>/python_tqt", because in the default location it may conflict with python-qt3 from a KDE3 installation.

The attached patch allows building python-tqt while sip4-tqt is located in a subfolder. In this way, we ensure that we do NOT use the distribution-provided SIP utility, if it exists?

As well as sip4-tqt, Python-tqt now being in a subfolder, it needs an (empty) file "__init__.py" too.

I use the following "configure" lines (you need to adapt to your environment):
mkdir build
cd build
python ../configure.py -c -n /usr/include/tqscintilla -q /usr/share/tqt3 -y tqt-mt -o /usr/lib64 -u -j 10 -d /usr/lib64/python2.7/site-packages/python_tqt -v /usr/share/sip/tqt -b /opt/trinity/bin 

Once installed, it gives the following files:

/usr/lib64/python2.7/site-packages/python_tqt/__init__.py
/usr/lib64/python2.7/site-packages/python_tqt/qt.so
/usr/lib64/python2.7/site-packages/python_tqt/qtcanvas.so
/usr/lib64/python2.7/site-packages/python_tqt/qtnetwork.so
/usr/lib64/python2.7/site-packages/python_tqt/qtsql.so
/usr/lib64/python2.7/site-packages/python_tqt/qttable.so
/usr/lib64/python2.7/site-packages/python_tqt/qtui.so
/usr/lib64/python2.7/site-packages/python_tqt/qtxml.so

And development files:
/usr/lib64/python2.7/site-packages/python_tqt/pyqtconfig.py
/usr/share/sip/tqt/pyqt-gpl.sip
/usr/share/sip/tqt/qt
/usr/share/sip/tqt/qt/copying.sip
/usr/share/sip/tqt/qt/qaccel.sip
[... and all other .sip files ...]

Comment 4 Francois Andriot 2013-12-29 03:48:34 CST

Created attachment 1788 [details]
python-trinity : fix detection of sip4-tqt and python-tqt

Now, let's build python-trinity.
Since it has been converted QT=>TQT and KDE=TDE, there is no risk of file conflict with anyone else. We can simply install this one in the default python module location (no subfolder needed).

Still, it needs to be able to detect the sip4-tqt and python-tqt. The attached patch fixes the detection of both.

Build command:
python configure.py -k /opt/trinity -L lib64 -v /usr/share/sip/trinity

Resulting files:

/usr/lib64/python2.7/site-packages/dcop.so
/usr/lib64/python2.7/site-packages/dcopexport.py
/usr/lib64/python2.7/site-packages/dcopext.py
/usr/lib64/python2.7/site-packages/pytdeconfig.py
/usr/lib64/python2.7/site-packages/tdeabc.so
/usr/lib64/python2.7/site-packages/tdecore.so
/usr/lib64/python2.7/site-packages/tdefile.so
/usr/lib64/python2.7/site-packages/tdefx.so
/usr/lib64/python2.7/site-packages/tdehtml.so
/usr/lib64/python2.7/site-packages/tdeio.so
/usr/lib64/python2.7/site-packages/tdemdi.so
/usr/lib64/python2.7/site-packages/tdeparts.so
/usr/lib64/python2.7/site-packages/tdeprint.so
/usr/lib64/python2.7/site-packages/tderesources.so
/usr/lib64/python2.7/site-packages/tdespell.so
/usr/lib64/python2.7/site-packages/tdeui.so
/usr/lib64/python2.7/site-packages/tdeutils.so

Development files:
/opt/trinity/bin/kdepyuic
/usr/share/sip/trinity/dcop/dcopclient.sip
/usr/share/sip/trinity/dcop/dcopmod.sip
/usr/share/sip/trinity/dcop/dcopobject.sip
/usr/share/sip/trinity/dcop/dcopref.sip
/usr/share/sip/trinity/dcop/dcopstub.sip
/usr/share/sip/trinity/dcop/typedefs.sip
[... and many other .sip files ...]

Comment 5 Francois Andriot 2013-12-29 04:25:42 CST

Created attachment 1789 [details]
pytdeextensions: convert QT to TQT in 'kdedistutils.py'

Building pytdeextensions is the most complicated of all our python stuff.
The most important file is "src/kdedistutils.py". It is used both at build time of pytdeextensions, and later at build time of next package (tde-guidance). It can also be used at runtime of non-compiled python scripts.

First, we need to finish the QT->TQT conversion (see attached patch).

Comment 6 Francois Andriot 2013-12-29 04:31:35 CST

Created attachment 1790 [details]
pytdeextensions: add sip4-tqt and python-tqt to the Python search path

Now, we must make the sip4-tqt and python-tqt available to the Python search path, so that calling applications (tde-guidance) do not need to worry about this.

In 'kdedistutils.py' we write the code at two location:
1) In the header of the script itself, so that python scripts that use kdedistutils will have the correct environment
2) In the "return_instance" function, which is later used to produce KCM modules (C++) from python scripts.

Comment 7 Francois Andriot 2013-12-29 04:39:23 CST

Created attachment 1791 [details]
pytdeextensions: adds the EXTRA_MODULES_DIR variable

Now it's the return of the EXTRA_MODULES_DIR variable (see bug #1188).
Since the tde-guidance will be installed in a non-standard location, but we don't know which yet, we need to add an environment variable-defined directory to the Python search path. The variable will be set later, when tde-guidance will be built.

Comment 8 Francois Andriot 2013-12-29 04:39:59 CST

Created attachment 1792 [details]
pytdeextensions: remove hardoded "lib" directory

Comment 9 Francois Andriot 2013-12-29 04:40:57 CST

Comment on attachment 1792 [details]
pytdeextensions: remove hardoded "lib" directory

Another patch to remove undesirable hardcoding in path.

Comment 10 Francois Andriot 2013-12-29 04:42:20 CST

Created attachment 1793 [details]
pytdeextensions: fix include directory location

Another patch to remove old, unwanted KDE include location, and instead use a more likely TDE include location.

Comment 11 Francois Andriot 2013-12-29 04:50:40 CST

Created attachment 1794 [details]
pytdeextensions: sample resulting 'kdedistutils.py' file

Now the funny stuff :-)
How to build this monster called "pytdeextensions".

At build time, to maximize the chances to have correct TDE-related path, I hardcode a lot of things in the "kdedistutils" script. Notice that there are already hardcoded things in the original script.


Here is the sed command:
In my case, the macros have value:
%{python_sitearch} = /usr/lib64/python2.7/site-packages
%{tde_libdir} = /opt/trinity/lib64
%{tde_tdeincludedir} = /opt/trinity/include/tde

The command is run in a loop:

for f in src/*.py; do
  sed -i "${f}" \
    -e "s|'sip-dir=','/usr/lib/pyshared/python\*'|'sip-dir=','%{python_sitearch}/sip4_tqt'|g" \
    -e "s|self.sip_dir = .*|self.sip_dir = \"%{python_sitearch}/sip4_tqt\"|g" \
    -e "s|'pyqt-dir=','/usr/lib/pyshared/python\*'|'pyqt-dir=','%{python_sitearch}/python_tqt'|g" \
    -e "s|self.pyqt_dir = .*|self.pyqt_dir = \"%{python_sitearch}/python_tqt\"|g" \
    -e "s|'pytde-dir=',None,|'pytde-dir=','%{python_sitearch}',|g" \
    -e "s|self.pytde_dir = None|self.pytde_dir = \"%{python_sitearch}\"|g" \
    -e "s|'kde-lib-dir=',None,|'kde-lib-dir=','%{tde_libdir}',|g" \
    -e "s|self.kde_lib_dir = None|self.kde_lib_dir = \"%{tde_libdir}\"|g" \
    -e "s|'kde-kcm-lib-dir=',None,|'kde-kcm-lib-dir=','%{tde_libdir}/trinity',|g" \
    -e "s|self.kde_kcm_lib_dir = None|self.kde_kcm_lib_dir = \"%{tde_libdir}/trinity\"|g" \
    -e "s|%{tde_includedir}/tde|%{tde_tdeincludedir}|g" \
    -e 's|"/kde"|"/tde"|' \
    -e 's|"-I" + self.kde_inc_dir + "/tde"|"-I/opt/trinity/include"|' \
    -e "s|/usr/lib/pyshared/python\*|%{python_sitearch}|g"
done

You can find a sample of the resulting file attached. All important variables are between lines 500-530.

Comment 12 Francois Andriot 2013-12-29 04:55:35 CST

After making sure that kdedistutils.py is correct, we can now build with

mkdir build
./setup.py build_libpythonize


./setup.py install \
	--root=/dev/shm/xxxxx \
	--prefix=/opt/trinity \
	--install-clib=/opt/trinity/lib64 \
	--install-cheaders=/opt/trinity/include/tde \
   -v

There are also some cleanup to do after this install step (moving files, deleting files ...). See ubuntu package "rules" file or RPM "spec" file for more details.

If you did not build correctly, you'll notice it at runtime afterward !

Comment 13 Francois Andriot 2013-12-29 04:58:04 CST

Created attachment 1795 [details]
tde-guidance: fix 'userconfig' for TDE R14

Now the interesting part: building tde-guidance !

The attached patch is a conversion for 'userconfig' utility so that it runs with TDE.

The Guidance package is also a hard piece to build. See ubuntu rules file or RPM spec files for details.

Finally, with all the previous steps, I have a working "tdecmshell userconfig" in TDE R14 !

Comment 14 Slávek Banko 2014-01-12 12:50:22 CST

Created attachment 1861 [details]
python-trinity : rename kdepyuic => tdepyuic

When testing the patch is necessary to think that 'tdepyuic' must be executable in the source folder.

Comment 15 Slávek Banko 2014-01-12 13:02:30 CST

Created attachment 1862 [details]
pytdeextensions: add sip4-tqt and python-tqt to the Python search path (1)

During my testing on Ubuntu Lucid I ran into a problem because the modules were installed in the /usr/lib/pymodules/python-2.6 => other path than that returned from get_python_lib.

I modified setting the paths in the way that they are searched in all modules folders. This also allow to remove the hard-coded paths sip_dir and pyqt_dir.

Comment 16 Slávek Banko 2014-01-12 13:11:07 CST

Created attachment 1863 [details]
pytdeextensions: remove hardoded "lib" directory (1)

The patch from attachment 1792 [details] on Debian / Ubuntu causes
FTBFS because install.prefix is /usr, while TDE is in /opt/trinity.

Instead of install.prefix I used self.install_prefix. To setup path to self.install_prefix I used ask_kde_config. This allow to remove additional hard-coded paths.

Comment 17 Slávek Banko 2014-01-12 13:14:37 CST

Created attachment 1864 [details]
pytdeextensions: fix include directory location (1)

By using self.install_prefix is possible to replace additional hard-coded paths.

Comment 18 Slávek Banko 2014-01-12 13:39:06 CST

Created attachment 1865 [details]
pytdeextensions: rename kdedistutils => tdedistutils

This patch assumes that previous pytdeextensions patches already applied.

Comment 19 Slávek Banko 2014-01-12 13:40:48 CST

Created attachment 1866 [details]
pytdeextensions: rename kdedesigner => tdedesigner

Now the latest patch for pytdeextensions.

Comment 20 Slávek Banko 2014-01-15 15:43:16 CST

Created attachment 1869 [details]
pytdeextensions: rename kdedistutils => tdedistutils (1)

Previous patch from attachment 1865 [details]
contained several typos ( " × ' ).

Comment 21 Slávek Banko 2014-01-15 15:54:47 CST

Created attachment 1870 [details]
tde-guidance: fix 'userconfig' for TDE R14 (1)

Finding paths adjusted to the same method as in the patch
from attachment 1862 [details].

Comment 22 Slávek Banko 2014-01-15 15:58:37 CST

Created attachment 1871 [details]
tde-guidance: rename kdedistutils => tdedistutils

Update tde-guidance to conform rename kdedistutils => tdedistutils.

Comment 23 Slávek Banko 2014-01-15 16:06:31 CST

François, please, can you test the updated patches? Or should I push patches to GIT and possible fixes we will deal after that?

Comment 24 Francois Andriot 2014-01-17 12:47:18 CST

Comment on attachment 1861 [details]
python-trinity : rename kdepyuic => tdepyuic

Patch is OK for me.

Comment 25 Francois Andriot 2014-01-17 13:10:29 CST

All other patches build and works for me.
You can push everything !

Comment 26 Slávek Banko 2014-01-18 09:00:28 CST

Comment on attachment 1787 [details]
python-tqt : fix detection of sip4-tqt

Pushed to GIT in hash bef77650.

Comment 27 Slávek Banko 2014-01-18 09:35:41 CST

Comment on attachment 1788 [details]
python-trinity : fix detection of sip4-tqt and python-tqt

Pushed to GIT in hash 9d39e811.

Comment 28 Slávek Banko 2014-01-18 09:36:07 CST

Comment on attachment 1861 [details]
python-trinity : rename kdepyuic => tdepyuic

Pushed to GIT in hash a2e9913e.

Comment 29 Slávek Banko 2014-01-18 11:58:35 CST

Comment on attachment 1789 [details]
pytdeextensions: convert QT to TQT in 'kdedistutils.py'

Pushed to GIT in hash 4997bbf9.

Comment 30 Slávek Banko 2014-01-18 11:59:08 CST

Comment on attachment 1862 [details]
pytdeextensions: add sip4-tqt and python-tqt to the Python search path (1)

Pushed to GIT in hash 7f3ecf70.

Comment 31 Slávek Banko 2014-01-18 11:59:30 CST

Comment on attachment 1791 [details]
pytdeextensions: adds the EXTRA_MODULES_DIR variable

Pushed to GIT in hash f664d2c4.

Comment 32 Slávek Banko 2014-01-18 11:59:50 CST

Comment on attachment 1863 [details]
pytdeextensions: remove hardoded "lib" directory (1)

Pushed to GIT in hash 6d061b32.

Comment 33 Slávek Banko 2014-01-18 12:00:12 CST

Comment on attachment 1864 [details]
pytdeextensions: fix include directory location (1)

Pushed to GIT in hash edae246d.

Comment 34 Slávek Banko 2014-01-18 12:00:30 CST

Comment on attachment 1869 [details]
pytdeextensions: rename kdedistutils => tdedistutils (1)

Pushed to GIT in hash 5559f44e.

Comment 35 Slávek Banko 2014-01-18 12:00:53 CST

Comment on attachment 1866 [details]
pytdeextensions: rename kdedesigner => tdedesigner

Pushed to GIT in hash d424535c.

Comment 36 Slávek Banko 2014-01-18 12:08:57 CST

Comment on attachment 1870 [details]
tde-guidance: fix 'userconfig' for TDE R14 (1)

Pushed to GIT in hash 605e1937.

Comment 37 Slávek Banko 2014-01-18 12:09:20 CST

Comment on attachment 1871 [details]
tde-guidance: rename kdedistutils => tdedistutils

Pushed to GIT in hash 0d63bc84.

Comment 38 Slávek Banko 2014-01-18 12:26:55 CST

It seems that all patches and also related changes in Debian / Ubuntu packaging are pushed into the GIT.

This bug report we can probably close.

Comment 39 David C. Rankin 2014-02-01 17:55:34 CST

(In reply to comment #38)
> It seems that all patches and also related changes in Debian / Ubuntu packaging
> are pushed into the GIT.
> 
> This bug report we can probably close.

Wait, hold off until we know this will work for everyone. I had R14 building OK on the 15th, let's see if it will build/operate before we close the bug.

Comment 40 Slávek Banko 2014-02-04 18:03:49 CST

Created attachment 1912 [details]
sip4-tqt : install as module by default

Although I announced that I'm not friend with Python, I tried to prepare patches for sip4-tqt and python-tqt that by default installed as Python modules.

This could simplify the building with default parameters. 

Please try it.

Comment 41 Slávek Banko 2014-02-04 18:04:43 CST

Created attachment 1913 [details]
python-tqt : install as module by default

Comment 42 Slávek Banko 2014-02-04 18:06:52 CST

François, please, can you check this patches?

Patches are prepared so that to the existing build scripts not need to be any changes.

Comment 43 Slávek Banko 2014-02-06 20:48:30 CST

Comment on attachment 1912 [details]
sip4-tqt : install as module by default

Pushed to GIT in hash 8cb66711

Comment 44 Slávek Banko 2014-02-06 20:48:56 CST

Comment on attachment 1913 [details]
python-tqt : install as module by default

Pushed to GIT in hash 4049988a

Comment 45 Slávek Banko 2014-02-08 12:08:29 CST

It seems that everything necessary has been done and that this bug report can be closed.

Comment 46 David C. Rankin 2014-02-08 14:05:32 CST

(In reply to comment #45)
> It seems that everything necessary has been done and that this bug report can
> be closed.

No,

  Where did the patches Go?

Comment 47 Darrell 2014-02-08 15:03:16 CST

The patches were pushed to git. Typically when patches are pushed we try in the bug report to remember to tag the patches as obsolete. That way people do not get confused and try to use a patch that is already pushed, which then will cause build failures and confusing messages about asking to reverse the patch.