In Drupal 8 Composer manager is obsolete and it is no longer needed. Instead of it, you can install libraries easier way.
Full instruction is available here https://www.drupal.org/node/2718229
But you can install required library from the module if it contains composer.json just in few steps:
1. Include Drupal repository into composer (need only once).
composer config repositories.drupal composer https://packages.drupal.org/8
2. Install library for your module (in my case it's module swiftmailer with version 8.x-1.0-alpha2)
composer require drupal/swiftmailer:1.0-alpha2
I hope that it was useful for you.