LDS Release 12.09

Major changes from previous stable release

• Clouddeck removed from LDS
• Compliance reports
• RBAC (Role Based Access Control)
• MAAS (Metal As A Service)
• Ubuntu 12.04 ("precise") and postgresql 9.1 support
• Package auto-upgrade profiles
• Saved searches
• New hardware information
• API
• Package repository management

Important changes, new features

RBAC

Role Based Access Control (RBAC) expands on the previously existing Access Groups feature. Administrators can now create roles and attach access groups, permissions and administrators to a role. Quick summary:

• role: the central object of RBAC. Contains permissions, access groups and administrators.
• access groups: administrative group of Landscape objects. Can contain computers, scripts, custom graphs, package profiles and others
• permissions: what the administrators in the role can do with the objects in the access groups

For example:

• role: DesktopAdmin

• access groups: desktop,laptop

• administrators: joe@example.com, martha@example.com

• permissions: Manage computer, Manage scripts, Manage pending computer

With this role, Joe and Martha can manage all the computers and scripts in the desktop and laptop access groups. They can also accept (and reject) pending computers into "desktop" or "laptop".

RBAC in LDS Upgrades

If you have administrators in access groups in a previous version of Landscape, new roles will be created by the upgrade process so that these administrators retain the privileges they had before.

For example, if you had an access group called Servers, and joe@example.com was an administrator in the access group:

• a new role called Role_Servers will be created during the LDS upgrade

• joe@example.com will be an administrator in this role

• the permissions Manage computer and View scripts will be assigned to the role

In this way, Joe will still be able to manage the computers in this access group after the upgrade.

MaaS

This release of LDS supports Canonical's MAAS (Metal as a Service), see https://wiki.ubuntu.com/ServerTeam/MAAS/ for details.

To configure a new Provisioning Server in Landscape, you will need:

• MAAS api credentials. Get them by going to your user preferences in the MAAS web interface

• API URL: it will be of the form https://<server>/MAAS/

Once that is done and you have a working MAAS installation, Landscape can be used to provision new machines that will become automatically registered in Landscape.

Auto upgrade profiles

You can create auto upgrade profiles which, when applied to computers via tags, can keep them up-to-date with all upgrades or just security upgrades. You can also select when you want to apply updates: if hourly or at specific times and days of the week.

API documentation

Please see the documentation link at the bottom of the LDS pages in your installation.

If you don't have LDS, then in on order to install the API client please add this PPA: landscape-api PPA

Alternatively, run this command:

sudo add-apt-repository ppa:landscape/landscape-api

To install the client:

sudo apt-get update
sudo apt-get install landscape-api

OpenID support

This release includes support for authenticating Landscape users with an external OpenID provider. To enable OpenID support, please add openid-provider-url and openid-logout-url to /etc/landscape/service.conf in the [landscape] section. For example:

[landscape]
(...)
openid-provider-url = https://login.ubuntu.com/
openid-logout-url = https://login.ubuntu.com/+logout

After making these changes, restart all Landscape services:

sudo lsctl restart

There is no provision yet to upgrade current users to OpenID authentication. If you want to change your existing installation to use OpenID, you will have to migrate the existing users manually.

Migrating existing users to OpenID authentication

To change the authentication mechanism of existing users to OpenID, you will need to insert each user's OpenID URL into the user entry in the database. Let's see an example.

Let's suppose we have an existing user called John Smith and we want to migrate him to OpenID. After changing /etc/landscape/service.conf and restarting the Landscape services, connect to the landscape-standalone-main database as an administrator:

ubuntu@ubuntu:~$ sudo -u postgres psql landscape-standalone-main
psql (8.4.11)
Type "help" for help.

landscape-standalone-main=# 

We now need to update the identity column of John's entry in the person table with his OpenID URL. Given John's email and his OpenID URL, the following SQL will do it:

UPDATE person SET identity = 'https://login.ubuntu.com/+id/FooBar' WHERE identity IS NULL AND email = 'john@example.com';
UPDATE 1
landscape-standalone-main=# 

This needs to be done for all users.

New hardware information

Starting with the 12.04.x client, hardware information in Landscape went through a major overhaul. Not only are new pieces of information available, the page has a better layout and all that information can be searched for either via the API or in the web UI. Here are some examples of the searches that are now possible:

• Disk size/product/serial

  • disk.size:[80gb TO 1tb] (supporting 80g(G) and 1t(T))

  • disk.size:40g (will grab 39g to 41g) (even fuzzier for RAM)

  • disk.product:string

• cpu.size:[1.7GHz TO *]

• cpu.vendor:intel

There are many more possibilities. This documentation still needs to be expanded.

APT backend replaces Smart

With this release of both LDS and the 12.04.x client, Landscape no longer uses Smart (http://smartpm.org) for package handling. Now, APT is used instead, both on the server and on the client. This means that package locks are no longer supported in Landscape, and will be replaced by package holds in apt/dpkg.

RUN_CRON setting

When doing more complex deployments, with several machines working as app and database servers, care must be taken to only run the Landscape cron jobs in one machine. In this case, select the machine where you want these jobs to run and set RUN_CRON="yes" in /etc/default/landscape-server on that machine only, leaving it disabled in all the others.

Repository Management

Here we introduce some concepts about Repository Management in Landscape.

Terminology

• The special pocket called release never gets mentioned in a suite. We will never have something called lucid-release and will just use lucid instead.

In a sources.list line, we would have:

deb http://archive.ubuntu.com/<distribution>/ <series>-<pocket> <component> [component ...]

Repository Management Getting Started

This is a quick tutorial showing how to mirror an upstream repository. For the sake of brevity, instead of mirroring the whole of, say, Lucid, we will just mirror the restricted component which is smaller.

We assume here that you have setup the API client already as explained elsewhere and that the landscape-api command is working.

   1 # import a secret GPG key. This will be used by LDS to sign the repository.
   2 # export a GPG secret key using gpg --export-secret-keys -a <keyid> > secret-key.pem
   3 # Note: the secret key must NOT have a passphrase. To remove the passphrase from a key,
   4 # use gpg --edit-key <keyid> before exporting it. See gpg(1) for details.
   5 $ landscape-api import-gpg-key secret-key secret-key.pem
   6 {u'fingerprint': u'5c49:8483:3dbf:5aaf:382e:000e:bc0e:d36a:1703:785b',
   7  u'has_secret': True,
   8  u'id': 7,
   9  u'name': u'secret-key'}
  10 
  11 # create a distribution
  12 $ landscape-api create-distribution ubuntu
  13 {u'creation_time': u'2011-10-14T19:44:13Z', u'name': u'ubuntu', u'series': []}
  14 
  15 # now create a series and some pockets, which are what hold the actual packages.
  16 # This will create a "lucid" series, with pockets for "release" and "updates"
  17 # of the "restricted" component, and for the i386 arch. It won't mirror any packages
  18 # yet.
  19 $ landscape-api create-series --pockets release,updates --components restricted --architectures i386 --gpg-key secret-key --mirror-uri http://archive.ubuntu.com/ubuntu/ --mirror-series lucid lucid ubuntu
  20 {u'creation_time': u'2011-10-14T19:44:57Z',
  21  u'name': u'lucid',
  22  u'pockets': [{u'apt_source_line': u'deb http://biriba.canonical.com/repository/standalone/ubuntu lucid restricted',
  23 (...)
  24 
  25 # now, let's sync the pockets. This will start the actual mirroring process:
  26 $ landscape-api sync-mirror-pocket release lucid ubuntu
  27 {u'children': [],
  28  u'computer_id': None,
  29  u'creation_time': u'2011-10-14T19:45:51Z',
  30  u'creator': {u'email': u'stan@example.com',
  31               u'id': 1,
  32               u'name': u'Stan Peters'},
  33  u'id': 101,
  34  u'parent_id': None,
  35  u'pocket_id': 5,
  36  u'pocket_name': u'release',
  37  u'progress': 0,
  38  u'summary': u"Sync pocket 'release' of series 'lucid' in distribution 'ubuntu'",
  39  u'type': u'SyncPocketRequest'}
  40 
  41 
  42 # the result of the above command is an activity, and it has an id.
  43 # We can query its progress by using get-activities with the activity id, and inspect the "progress" result, which is a percentage:
  44 $ landscape-api get-activities --query id:101
  45 (...)
  46   u'id': 101,
  47   u'parent_id': None,
  48   u'pocket_id': 5,
  49   u'pocket_name': u'release',
  50   u'progress': 35,
  51 (...)
  52 
  53 # it's almost done. We can only issue another sync-mirror-pocket call once the above is finished.
  54 # Once it's finished, we can issue another call, this time to sync the updates pocket:
  55 $ landscape-api sync-mirror-pocket updates lucid ubuntu
  56 (...)
  57  u'id': 102,
  58 (...)
  59 
  60 # while the sync happens, we can create a repository profile which we will later apply to computers:
  61 $ landscape-api create-repository-profile --description "This profile is for LDS servers." lds-profile
  62 {u'all_computers': False,
  63  u'description': u'This profile is for LDS-servers.',
  64  u'id': 5,
  65  u'name': u'lds-profile',
  66  u'pockets': [],
  67  u'tags': []}
  68 
  69 # now we associate computers with the tag "lds" to this repository profile:
  70 $ landscape-api associate-repository-profile --tags lds lds-profile
  71 {u'all_computers': False,
  72  u'description': u'This-profile-is-for-LDS-servers.',
  73  u'id': 5,
  74  u'name': u'lds-profile',
  75  u'pockets': [],
  76  u'tags': [u'lds']}
  77 
  78 # finally, we say which pockets are part of this repository profile:
  79 $ landscape-api add-pockets-to-repository-profile lds-profile release,updates lucid ubuntu
  80 True

At the end of the above script, computers with the "lds" tag will get an entry in /etc/apt/sources.list.d/ pointing to our newly created release and updates repository for Lucid restricted component.

The repositories are also visible with a web browser at these URLs: http://<lds-server>/repository/standalone/ubuntu/pool/ and http://<lds-server>/repository/standalone/ubuntu/dists/

Repository Management Mirroring Tips

Here are some simple tips showing how to create some basic and useful standard repositories using Landscape. They all revolve around the create-pocket API call, so you should already have at least created a distribution (like ubuntu) and a series (like precise).

The basic usage for create-pocket is like this:

• mirror-suite: what you want to mirror:

  • SERIES: mirror SERIES as it was released, without any updates. For example: precise

  • SERIES-updates: mirror the updates for SERIES. For example: precise-updates

  • SERIES-security: mirror the security updates for SERIES. For example: precise-security

• pocket name: better to stick with the suite above. For example, if you are mirroring SERIES-updates, call the pocket updates. If mirroring a release, i.e., no suffix in the mirror-suite above, use release for the name.

• components: a simple comma-separated list of the components you want to mirror. Example: main,restricted.

• architectures: a simple comma-separated list of the architectures you want to mirror. Example: i386,amd64

• mirror key: even though it's a mirror, Landscape will resign it with its own key. Therefore you must upload a private passphrase-less GPG key to Landscape and use its name here.

Here are some examples of how to use create-pocket:

• Mirror all security updates for precise for all components and architectures:

landscape-api create-pocket --mirror-suite precise-security --mirror-uri http://archive.ubuntu.com/ubuntu/ security precise ubuntu main,restricted,universe,multiverse i386,amd64 mirror mirror-sign-key

This is how the sources.list entry will look:

deb http://lds.example.com/repository/standalone/ubuntu precise-security main restricted universe multiverse

• Mirror lucid normal updates for main and universe, i386 only:

landscape-api create-pocket --mirror-suite lucid-updates --mirror-uri http://archive.ubuntu.com/ubuntu/ updates lucid ubuntu main,universe i386 mirror mirror-sign-key

This is how the sources.list entry will look:

deb http://lds.example.com/repository/standalone/ubuntu lucid-updates main universe

• Mirror precise as released, without any updates, for main and universe, i386 only:

landscape-api create-pocket --mirror-suite precise --mirror-uri http://archive.ubuntu.com/ubuntu/ release precise ubuntu main,universe i386 mirror mirror-sign-key

This is how the sources.list entry will look:

deb http://lds.example.com/repository/standalone/ubuntu precise main universe

Be sure to call sync-mirror-pocket to start the mirroring process:

landscape-api sync-mirror-pocket release precise ubuntu

Upload pockets

Landscape can also manage repositories which hold packages that are uploaded to it by authorised users. Here is a quick howto for creating and uploading packages to such a repository.

Let's say we cant to create a staging area where some users can upload packages to. Assuming it's for precise and that the ubuntu distribution is created already, this would do it:

landscape-api create-pocket staging precise ubuntu main i386 upload upload-sign-key

• staging: the name of our upload pocket

• ubuntu: the distribution (create-distribution ubuntu)

• precise: the series (create-series precise ubuntu)

• main: the component

• i386: the architecture

• upload: the pocket type

• upload-sign-key: a private passphrase-less GPG key imported with gpg-import-key

Such a repository will be accessible via this sources.list entry:

deb http://lds.example.com/repository/standalone/ubuntu precise-staging main

You can choose who is allowed to upload packages to this pocket. Since the option --upload-allow-unsigned was not used when creating the pocket, only uploads signed by any of the uploader gpg keys will be allowed. Unsigned uploads, or signed by a key not in that list, will be rejected. To add or remove a key from that list, use add-uploader-gpg-keys-to-pocket and remove-uploader-gpg-keys-from-pocket respectively.

To upload packages to this pocket we use the tool dput with this configuration section in ~/.dput.cf:

[lds]
fqdn = lds.example.com
method = https
incoming = /upload/standalone/%(lds)s

• The %(lds)s bit will be replaced by whatever follows the lds: prefix in the dput command shown below

The package debian/changelog file must contain the precise-staging target, like this example:

my-package (1.0-0ubuntu1) precise-staging; urgency=low

  * Released 1.0

  -- Package Builder <builder@example.com>  Thu, 30 Aut 2012 14:57:05 -0300
(...)

Since Landscape doesn't build packages from source, you will have to build the package locally for the right architecture and then upload the binary changes file, like this:

$ dput lds:ubuntu/precise/staging my-package_1.0-0ubuntu1_i386.changes

Note the path components: lds:distro/series/pocket.

If there are errors, they will be logged in the server's /var/log/landscape/package-upload-1.log log file. An email will also be sent to the uploader detailing the error.

For example, let's say you forgot to add the public GPG key of a developer to this pocket. If he tries to upload a package, he will get an error like this back in an email:

Subject: Package import failed for 'storm_0.18.1-0landscape4_i386.changes'

The following error(s) occured in package import:

* Uploaded data is not signed, but the destination pocket requires it
* The signature on following file(s) could not be verified.  Please make sure to import the GPG key(s) used to sign the files or use a different key to sign them.
  - storm_0.18.1-0landscape4_i386.changes (key id 784259B3F3DDC290)

To fix this, we authorize uploads signed by that GPG key:

   1 # Key exported to a file using gpg -a --export 784259B3F3DDC290 > 784259B3F3DDC290.pem
   2 $ lsapi import-gpg-key key-f3ddc290 ./784259B3F3DDC290.pem
   3 {u'fingerprint': u'6f96:333f:ae2c:0ce5:254a:8742:7842:59b3:f3dd:c290',
   4  u'has_secret': False,
   5  u'id': 7,
   6  u'name': u'key-f3ddc290'}
   7 
   8 # authorize it
   9 $ lsapi add-uploader-gpg-keys-to-pocket staging precise ubuntu key-f3ddc290
  10 (...)
  11  u'upload_allow_unsigned': False,
  12  u'upload_gpg_keys': [{u'fingerprint': u'6f96:333f:ae2c:0ce5:254a:8742:7842:59b3:f3dd:c290',
  13                        u'has_secret': False,
  14                        u'id': 7,
  15                        u'name': u'key-f3ddc290'}]}

Clouddeck no longer available

Clouddeck is no longer included with LDS. Starting with this release, there is no longer a "cloud" top level menu. Instances launched through Clouddeck that became Landscape registered computers will still be visible, but no cloud actions can be taken on them.

Impact on existing data

All your ssh keys, security groups, etc, will remain in the cloud, but they will all have mangled names. For example, if you had an ssh security group called ssh in Clouddeck/Landscape, its name in the backend cloud might be something like 1-ssh-Z9ekPFox. There is no provision in Landscape to show this mapping.

Clouddeck-only users, that is, users who were invited via the settings tab on the cloud page and had only cloud related privileges, will still exist after the upgrade, but without ties to a Landscape account. If you want them to become Landscape administrators, invite them as usual using the Administrators page in Landscape.

Upgrading

You can now select the version of LDS you want to upgrade to by visiting your account at https://landscape.canonical.com/:

After you have made the selection and clicked on "Change target release", your LDS installation will already be pointed at the repository containing the new version. Canonical supports the last two major versions of Landscape released, meaning you do not have to perform a major upgrade more than once a year -- or until you choose to deploy new functionality.

Quickstart upgrade

If you have the landscape-server-quickstart package installed and are using just one server to host LDS and the Database, then you have what it's called a quickstart setup. In this case, to upgrade to the new version of LDS, just select the version you want as described above and run:

sudo apt-get update && sudo apt-get dist-upgrade

• Reply with "N" to any questions dpkg may ask about configuration files. Do not replace any existing configuration file with a version from the package.

Due to a minor bug in the upgrade script, there is one manual change that is necessary. Please change the line stores in the [maintenance] section of /etc/landscape/service.conf to also include the session-autocommit store, so that the line reads like this:

[maintenance]
...
stores = main account-1 resource-1 package session knowledge session-autocommit
...

There is no need to restart any service after this change.

Non-quickstart upgrade

If you don't have the landscape-server-quickstart package installed, then it's called a non-quickstart setup. In this case, please follow these steps to perform the upgrade:

• stop all landscape and clouddeck services on all machines

• on the database server, install the package python-apt:

sudo apt-get install python-apt

From now on, the instructions apply to all application server machines.

• add the following line to the [maintenance] section of /etc/landscape/service.conf:

diskstore = /var/lib/landscape/landscape-disk-store

• in the same [maintenance] section, change the stores line to include the session-autocommit store, so it reads like this:

stores = stores = main account-1 resource-1 package session knowledge session-autocommit

• add this new section to /etc/landscape/service.conf:

[package-upload]
stores = main account-1
threads = 10
port = 9100
root-url = http://localhost:9100
mailer = queue
mailer-path = /var/lib/landscape/landscape-mail-queue-1

• add a new [load-shaper] section to /etc/landscape/service.conf:

[load-shaper]
critical-load = 10.0
good-load = 3.0
good-duration = 60

• remove the existing [statsd] section and its contents entirely from /etc/landscape/service.conf

• add these lines to the existing [landscape] section of /etc/landscape/service.conf:

reprepro-binary = /opt/canonical/landscape/scripts/reprepro-wrapper.sh
repository-path = /var/lib/landscape/landscape-repository

• In the apache config file for LDS, in the https (port 443) vhost section, remove the final /message-system part from the existing rewrite rule. This:

    RewriteRule ^/message-system http://localhost:8090/++vh++https:${hostname}:443/++/message-system [P,L]

Should become this:

    RewriteRule ^/message-system http://localhost:8090/++vh++https:${hostname}:443/++/ [P,L]

• Also in the https (port 443) vhost section, add a new rewrite rule for script attachments and package upload, just below the existing /api rule, and remove the cloud rule:

(...)
    RewriteRule ^/api http://localhost:9080/ [P,L]
    RewriteRule ^/attachment/(.*) http://localhost:8090/attachment/$1 [P,L] <--- add
    RewriteRule ^/upload/(.*) http://localhost:9100/$1 [P,L] <--- add
    #RewriteRule ^/cloud/(.*) http://localhost:8600/$1 [P,L] <--- remove
(...)

• visit your account page at https://landscape.canonical.com/ and select the new version of LDS as explained earlier

• upgrade using apt-get update and apt-get dist-upgrade. As usual, reply with "N" when asked about overwriting any Landscape configuration file. If you don't have UPGRADE_SCHEMA enabled in /etc/default/landscape-server, the services will fail to restart at the end of the upgrade. That's normal.

• restart apache so it gets the new landscape group membership:

sudo /etc/init.d/apache2 restart

• if automatic schema upgrades are disabled (see /etc/default/landscape-server), then you will also need to manually upgrade the Landscape schema:

sudo setup-landscape-server

• Now restart the landscape services (lsctl is a new helper that will act on all enabled landscape services):

sudo lsctl restart

Quickstart upgrade from Beta3

If you have the landscape-server-quickstart package installed and are using just one server to host LDS and the Database, then you have what it's called a quickstart setup. In this case, to upgrade to the new version of LDS, just just select the version of LDS you want as explained earlier in this document and run:

sudo apt-get update && sudo apt-get dist-upgade

• Reply with "N" to any questions dpkg may ask about configuration files. Do not replace any existing configuration file with a version from the package.

Non-quickstart upgrade from Beta3

There are no configuration file changes necessary if you are upgrading from Beta3. Just follow these steps:

• stop all landscape and clouddeck services on all machines
• select the version of LDS you want as explained earlier in this document

• upgrade using apt-get update and apt-get dist-upgrade. As usual, reply with "N" when asked about overwriting any Landscape configuration file.

• restart apache so it gets the new landscape group membership:

sudo /etc/init.d/apache2 restart

• manually upgrade the clouddeck schema:

sudo /etc/init.d/clouddeck stop
sudo clouddeck-schema /etc/clouddeck/stores.cfg

• if automatic schema upgrades are disabled (see /etc/default/landscape-server), then you will also need to manually upgrade the Landscape schema:

sudo setup-landscape-server

• Now start the landscape services (lsctl is a new helper that will act on all enabled landscape services):

sudo lsctl start

Lucid to Precise Release Upgrade

Please follow these instructions: Lucid to Precise Release Upgrade

Fresh installation

Quickstart

For a quickstart deployment, please follow the instructions given to you on your Landscape Hosted account (https://landscape.canonical.com/account/<your-account>/how-to-get-dedicated, replace <your-account> with your Landscape account name).

Non-quickstart

See the Recommended deployment for LDS 12.09 document.

Known Issues

Here we describe some of the known issues that customers might encounter while using this release.

RBAC and Pending Computers

Restricted administators, even the ones with the permission to manage pending computers, won't see or get a "pending computer" alert. If you have the permission to manage pending computers, you will have to manually navigate to the page where they are listed: https://landscape.example.com/account/standalone/pending-computers

Error about pycurl not being installed

If you are installing LDS on a really minimal Ubuntu Server installation, you might get an error about the python module pycurl not being installed. If that's the case, please install it and continue with the installation:

sudo apt-get install python-pycurl

Depending on which step the error happened, you may need to run dpkg-reconfigure or apt-get -f install.

Backtrace in maintenance logs about a missing store

The quickstart upgrade isn't updating the stores definition in the [maintenance] section in /etc/landscape/service.conf. As a result, the maintenance cron job (which runs daily) will contain a backtrace like this:

Traceback (most recent call last):
  File "./maintenance", line 7, in <module>
    canonical.landscape.scripts.maintenance.run()
(...)
  File "/usr/lib/python2.6/dist-packages/storm/zope/zstorm.py", line 146, in create
    raise ZStormError("Store named '%s' not found" % name)
storm.zope.interfaces.ZStormError: Store named 'session-autocommit' not found

To fix this, just edit /etc/landscape/service.conf and change the stores line under the [maintenance] section to look like this:

[maintenance]
...
stores = main account-1 resource-1 package session knowledge session-autocommit
...

There is no need to restart any service after this change.

System users and groups in LDAP or other NSS sources

The permissions on the /opt/canonical/landscape directory were changed to 0750 root:landscape. The apache user still needs to read that content, though, so the landscape package tries to add the www-data user to the landscape group. If you have the system users and/or groups in LDAP or other NSS source, particularly the landscape group, this will fail silently (the command we run is usermod -G landscape www-data).

If that's the case, you will have to manually add the www-data user to the landscape group that is in your NSS source.