Neutron扩展资源API流程分析

发表于 2017-12-15   |   分类于 技术

neutron除了核心资源以外还有扩展资源,核心资源的API处理实现流程已经分析过了(见:https://blog.try-except.com/technology/neutron-server-api.html)。
那么neutron是如何支持扩展资源的API并进行相关的路由配置呢?

其实答案也在paste-api.conf文件中,其中有这样一段配置:

[filter:extensions]
paste.filter_factory = neutron.api.extensions:plugin_aware_extension_middleware_factory

这个extensions的filter在neutron-server启动的时候会被加载,先看一下它的构造方法:

def plugin_aware_extension_middleware_factory(global_config, **local_config):
    """Paste factory."""
    def _factory(app):
        ext_mgr = PluginAwareExtensionManager.get_instance()
        return ExtensionMiddleware(app, ext_mgr=ext_mgr)
    return _factory

构造了一个PluginAwareExtensionManager对象,并用它继续构造了一个ExtensionMiddleware对象。对于PluginAwareExtensionManager,我们有必要看一下它的创建过程get_instance()

    @classmethod
    def get_instance(cls):
        if cls._instance is None:
            service_plugins = directory.get_plugins()
            cls._instance = cls(get_extensions_path(service_plugins),
                                service_plugins)
        return cls._instance

很容易看出来这其实是一个单例模式的构造过程。

service_plugins = directory.get_plugins()

这一句是获取到了所有已经加载了的service plugin对象,至于service plugin对象是怎么加载的,我会另写一篇文章介绍,这里避免内容过多就不再叙说。

get_extensions_path(service_plugins)

这一句也是一个关键,根据加载的service plugin获取extension的搜索路径,为什么获取extensino的存放路径要用到service plugin呢?

def get_extensions_path(service_plugins=None):
    paths = collections.OrderedDict()

    # Add Neutron core extensions
    paths[core_extensions.__path__[0]] = 1
    if service_plugins:
        # Add Neutron *-aas extensions
        for plugin in service_plugins.values():
            neutron_mod = provider_configuration.NeutronModule(
                plugin.__module__.split('.')[0])
            try:
                paths[neutron_mod.module().extensions.__path__[0]] = 1
            except AttributeError:
                # Occurs normally if module has no extensions sub-module
                pass

    # Add external/other plugins extensions
    if cfg.CONF.api_extensions_path:
        for path in cfg.CONF.api_extensions_path.split(":"):
            paths[path] = 1

    LOG.debug("get_extension_paths = %s", paths)

    # Re-build the extension string
    path = ':'.join(paths)
    return path

首先创建了一个OrderedDict,也就是一个有序的字典对象paths

paths[core_extensions.__path__[0]] = 1

这句话向paths字典添加了核心资源的扩展路径,core_extensions就是neutron.extensions模块

from neutron import extensions as core_extensions

所以core_extensions.__path__[0]就是neutron.extensions的真实路径。接下来遍历的所有的service_plugins。

neutron_mod = provider_configuration.NeutronModule(
                plugin.__module__.split('.')[0])

上面这句话中plugin.__module__.split('.')[0]这句话的意思是取plugin模块名中第一个点之前的名称。举个例子:如果一个service plugin的代码实现是在neutron这个大模块下面的,例如以下的service plugin:

[neutron.service_plugins]
auto_allocate = neutron.services.auto_allocate.plugin:Plugin
dummy = neutron.tests.unit.dummy_plugin:DummyServicePlugin
flavors = neutron.services.flavors.flavors_plugin:FlavorsPlugin
loki = neutron.services.loki.loki_plugin:LokiPlugin
metering = neutron.services.metering.metering_plugin:MeteringPlugin
qos = neutron.services.qos.qos_plugin:QoSPlugin
revisions = neutron.services.revisions.revision_plugin:RevisionPlugin
router = neutron.services.l3_router.l3_router_plugin:L3RouterPlugin
segments = neutron.services.segments.plugin:Plugin
tag = neutron.services.tag.tag_plugin:TagPlugin
timestamp = neutron.services.timestamp.timestamp_plugin:TimeStampPlugin
trunk = neutron.services.trunk.plugin:TrunkPlugin

它们的模块名称第一个点之前的都是neutron,那么plugin.__module__.split('.')[0]的值就是neutron。
provider_configuration.NeutronModule对象的module()返回的就是构造时传入的module名对应的module对象。

            try:
                paths[neutron_mod.module().extensions.__path__[0]] = 1
            except AttributeError:
                # Occurs normally if module has no extensions sub-module
                pass

所以neutron_mod.module().extensions.__path__[0]指的就是service plugin顶层模块下面的extensions模块的路径,例如:neutron.extensions模块的路径。
那么同理,像下面所列的service plugin:

[neutron.service_plugins]
firewall = neutron_fwaas.services.firewall.fwaas_plugin:FirewallPlugin
firewall_v2 = neutron_fwaas.services.firewall.fwaas_plugin_v2:FirewallPluginV2

它们的extensions路径就是模块neutron_fwaas.extensinos对应的路径了。

这里就明白了,为啥获取extension的搜索路径要通过service_plugin,就是需要通过service plugin的模块名来计算对应的path。

除此以外,用户也可以在配置文件中指定其他的路径地址,也会被加到paths字典中。

    if cfg.CONF.api_extensions_path:
        for path in cfg.CONF.api_extensions_path.split(":"):
            paths[path] = 1

最后所有的路径会用冒号连接并返回。

path = ':'.join(paths)
return path

类似CLASSPATH一样。

继续往下:

cls._instance = cls(get_extensions_path(service_plugins),
                                service_plugins)

返回的路径和所有的service plugin对象再被用来创建extension manager:PluginAwareExtensionManager对象。
接下来就来看一下ExtensionMiddleware对象的创建,其__init__方法如下:

    def __init__(self, application,
                 ext_mgr=None):
        self.ext_mgr = (ext_mgr
                        or ExtensionManager(get_extensions_path()))
        mapper = routes.Mapper()

        # extended resources
        for resource in self.ext_mgr.get_resources():
            path_prefix = resource.path_prefix
            if resource.parent:
                path_prefix = (resource.path_prefix +
                               "/%s/{%s_id}" %
                               (resource.parent["collection_name"],
                                resource.parent["member_name"]))

            LOG.debug('Extended resource: %s',
                      resource.collection)
            for action, method in six.iteritems(resource.collection_actions):
                conditions = dict(method=[method])
                path = "/%s/%s" % (resource.collection, action)
                with mapper.submapper(controller=resource.controller,
                                      action=action,
                                      path_prefix=path_prefix,
                                      conditions=conditions) as submap:
                    submap.connect(path_prefix + path, path)
                    submap.connect(path_prefix + path + "_format",
                                   "%s.:(format)" % path)

            for action, method in resource.collection_methods.items():
                conditions = dict(method=[method])
                path = "/%s" % resource.collection
                with mapper.submapper(controller=resource.controller,
                                      action=action,
                                      path_prefix=path_prefix,
                                      conditions=conditions) as submap:
                    submap.connect(path_prefix + path, path)
                    submap.connect(path_prefix + path + "_format",
                                   "%s.:(format)" % path)

            mapper.resource(resource.collection, resource.collection,
                            controller=resource.controller,
                            member=resource.member_actions,
                            parent_resource=resource.parent,
                            path_prefix=path_prefix)

        # extended actions
        action_controllers = self._action_ext_controllers(application,
                                                          self.ext_mgr, mapper)
        for action in self.ext_mgr.get_actions():
            LOG.debug('Extended action: %s', action.action_name)
            controller = action_controllers[action.collection]
            controller.add_action(action.action_name, action.handler)

        # extended requests
        req_controllers = self._request_ext_controllers(application,
                                                        self.ext_mgr, mapper)
        for request_ext in self.ext_mgr.get_request_extensions():
            LOG.debug('Extended request: %s', request_ext.key)
            controller = req_controllers[request_ext.key]
            controller.add_handler(request_ext.handler)

        self._router = routes.middleware.RoutesMiddleware(self._dispatch,
                                                          mapper)
        super(ExtensionMiddleware, self).__init__(application)

一开始就见到了熟悉的mapper对象,所以可以确定路由设定肯定是在这里。
参数application就是用来创建middleware的WSGI app。
参数ext_mgr是上面已经创建好的PluginAwareExtensionManager对象。

现在我们需要知道extensino扩展有三种情况

  • 扩展新的资源:extended resources
  • 对已经存在的资源扩展新的动作:extended actions
  • 对某个请求增加参数:extended requests

extended resources

相关代码:

        # extended resources
        for resource in self.ext_mgr.get_resources():
            path_prefix = resource.path_prefix
            if resource.parent:
                path_prefix = (resource.path_prefix +
                               "/%s/{%s_id}" %
                               (resource.parent["collection_name"],
                                resource.parent["member_name"]))

            LOG.debug('Extended resource: %s',
                      resource.collection)
            for action, method in six.iteritems(resource.collection_actions):
                conditions = dict(method=[method])
                path = "/%s/%s" % (resource.collection, action)
                with mapper.submapper(controller=resource.controller,
                                      action=action,
                                      path_prefix=path_prefix,
                                      conditions=conditions) as submap:
                    submap.connect(path_prefix + path, path)
                    submap.connect(path_prefix + path + "_format",
                                   "%s.:(format)" % path)

            for action, method in resource.collection_methods.items():
                conditions = dict(method=[method])
                path = "/%s" % resource.collection
                with mapper.submapper(controller=resource.controller,
                                      action=action,
                                      path_prefix=path_prefix,
                                      conditions=conditions) as submap:
                    submap.connect(path_prefix + path, path)
                    submap.connect(path_prefix + path + "_format",
                                   "%s.:(format)" % path)

            mapper.resource(resource.collection, resource.collection,
                            controller=resource.controller,
                            member=resource.member_actions,
                            parent_resource=resource.parent,
                            path_prefix=path_prefix)

与核心资源不同,核心资源的资源是固定的,所以neutron通过一个字典数据结构将这些资源罗列了出来:

RESOURCES = {'network': 'networks',
             'subnet': 'subnets',
             'subnetpool': 'subnetpools',
             'port': 'ports'}

而扩展资源由于用户可以自定义扩展,无法提前预知,所以需要动态的获取:

for resource in self.ext_mgr.get_resources():

这里调用了ext_mgr的get_resources方法来获取的:

    def get_resources(self):
        """Returns a list of ResourceExtension objects."""
        resources = []
        resources.append(ResourceExtension('extensions',
                                           ExtensionController(self)))
        for ext in self.extensions.values():
            resources.extend(ext.get_resources())
        return resources

resources列表先追加了一个ResourceExtension类型的资源,这就是扩展了一个资源,它的collection名就是extensions。相当与实现了/v2.0/extensions。
剩下的资源获取都是通过遍历了ext_mgr中的所有的extension,调用了每个extension对象的get_resources方法。所以需要先看一下extension是怎么被加载到extension manager中的。主要方法是其中的_load_all_extensions且是在extension manager初始化的时候被加载的:

    def __init__(self, path):
        LOG.info(_LI('Initializing extension manager.'))
        self.path = path
        self.extensions = {}
        self._load_all_extensions()

这里的path就是前面所讲的extension模块的路径集合,路径和路径之间用冒号连接。加载extension是调用了_load_all_extensions方法:

    def _load_all_extensions(self):
        for path in self.path.split(':'):
            if os.path.exists(path):
                self._load_all_extensions_from_path(path)
            else:
                LOG.error(_LE("Extension path '%s' doesn't exist!"), path)

遍历了所有的extension路径,如果路径存在,则对这些路径调用_load_all_extensions_from_path方法:

    def _load_all_extensions_from_path(self, path):
        for f in sorted(os.listdir(path)):
            try:
                LOG.debug('Loading extension file: %s', f)
                mod_name, file_ext = os.path.splitext(os.path.split(f)[-1])
                ext_path = os.path.join(path, f)
                if file_ext.lower() == '.py' and not mod_name.startswith('_'):
                    mod = imp.load_source(mod_name, ext_path)
                    ext_name = mod_name[0].upper() + mod_name[1:]
                    new_ext_class = getattr(mod, ext_name, None)
                    if not new_ext_class:
                        LOG.warning(_LW('Did not find expected name '
                                        '"%(ext_name)s" in %(file)s'),
                                    {'ext_name': ext_name,
                                     'file': ext_path})
                        continue
                    new_ext = new_ext_class()
                    self.add_extension(new_ext)
            except Exception as exception:
                LOG.warning(_LW("Extension file %(f)s wasn't loaded due to "
                                "%(exception)s"),
                            {'f': f, 'exception': exception})

在这个方法中遍历了指定路径下的所有文件,当文件名的后缀是".py"并且文件名不以"_"开头,就会去加载这个py文件。加载完成后会去find与文件名同名(首字母大写)的类,这个类就是需要加载的extension类。如果找到了就会创建这个类的对象,并通过add_extension方法将这个extension对象保存到manager对像中。
这样就完成了所有extension的加载。
回到get_resources方法:

    def get_resources(self):
        """Returns a list of ResourceExtension objects."""
        resources = []
        resources.append(ResourceExtension('extensions',
                                           ExtensionController(self)))
        for ext in self.extensions.values():
            resources.extend(ext.get_resources())
        return resources

调用完每个extension对象的get_resource方法后把结果保存到列表中返回,这个列表中的每个对象都应该是ResourceExtension类型的。
后面的操作就是遍历每一个resource对象,利用mapper给每个resource做路由设定。

所以,扩展新的资源,需要在对应extension的get_resources方法中返回一个ResourceExtension对象。

extended actions

相关代码如下:

        # extended actions
        action_controllers = self._action_ext_controllers(application,
                                                          self.ext_mgr, mapper)
        for action in self.ext_mgr.get_actions():
            LOG.debug('Extended action: %s', action.action_name)
            controller = action_controllers[action.collection]
            controller.add_action(action.action_name, action.handler)

首先通过_action_ext_controllers创建了一个collection和controller对应关系的字典:

    def _action_ext_controllers(self, application, ext_mgr, mapper):
        """Return a dict of ActionExtensionController-s by collection."""
        action_controllers = {}
        for action in ext_mgr.get_actions():
            if action.collection not in action_controllers.keys():
                controller = ActionExtensionController(application)
                mapper.connect("/%s/:(id)/action.:(format)" %
                               action.collection,
                               action='action',
                               controller=controller,
                               conditions=dict(method=['POST']))
                mapper.connect("/%s/:(id)/action" % action.collection,
                               action='action',
                               controller=controller,
                               conditions=dict(method=['POST']))
                action_controllers[action.collection] = controller

        return action_controllers

通过extension manager的get_actions方法获取到所有扩展的action。

    def get_actions(self):
        """Returns a list of ActionExtension objects."""
        actions = []
        for ext in self.extensions.values():
            actions.extend(ext.get_actions())
        return actions

类似的,也是依次调用了所有的extension的get_actions方法,并将所有结果组成一个list返回。
然后遍历每个action,为每个action都创建了一个controller:ActionExtensionController

class ActionExtensionController(wsgi.Controller):

    def __init__(self, application):
        self.application = application
        self.action_handlers = {}

    def add_action(self, action_name, handler):
        self.action_handlers[action_name] = handler

    def action(self, request, id):
        input_dict = self._deserialize(request.body,
                                       request.get_content_type())
        for action_name, handler in six.iteritems(self.action_handlers):
            if action_name in input_dict:
                return handler(input_dict, request, id)
        # no action handler found (bump to downstream application)
        response = self.application
        return response

可以看到每个action name对应一个handler,处理的时候会根据请求调用对应的handler。
之后为每个action添加了两条路由:

                mapper.connect("/%s/:(id)/action.:(format)" %
                               action.collection,
                               action='action',
                               controller=controller,
                               conditions=dict(method=['POST']))
                mapper.connect("/%s/:(id)/action" % action.collection,
                               action='action',
                               controller=controller,
                               conditions=dict(method=['POST']))

controller指定为上面创建的controller对象,action方法名定义为"action"。这样,当请求过来的时候,根据这两条路由,就能执行到上述controller的action方法了。
当所有的action保存到字典后,返回。返回之后遍历所有的action,对每个action执行以下代码:

            controller = action_controllers[action.collection]
            controller.add_action(action.action_name, action.handler)

为action对应的controller添加需要扩展的方法名和handler,这样就完成了扩展。

extended requests

扩展的代码如下:

        req_controllers = self._request_ext_controllers(application,
                                                        self.ext_mgr, mapper)
        for request_ext in self.ext_mgr.get_request_extensions():
            LOG.debug('Extended request: %s', request_ext.key)
            controller = req_controllers[request_ext.key]
            controller.add_handler(request_ext.handler)

流程和action扩展十分类似,先通过_request_ext_controllers获取到一个request扩展的key和controller对应关系的字典。

    def _request_ext_controllers(self, application, ext_mgr, mapper):
        """Returns a dict of RequestExtensionController-s by collection."""
        request_ext_controllers = {}
        for req_ext in ext_mgr.get_request_extensions():
            if req_ext.key not in request_ext_controllers.keys():
                controller = RequestExtensionController(application)
                mapper.connect(req_ext.url_route + '.:(format)',
                               action='process',
                               controller=controller,
                               conditions=req_ext.conditions)

                mapper.connect(req_ext.url_route,
                               action='process',
                               controller=controller,
                               conditions=req_ext.conditions)
                request_ext_controllers[req_ext.key] = controller

        return request_ext_controllers

对每个request extension都创建了一个RequestExtensionController对象,然后通过mapper进行路由的绑定。看一下这个controller:

class RequestExtensionController(wsgi.Controller):

    def __init__(self, application):
        self.application = application
        self.handlers = []

    def add_handler(self, handler):
        self.handlers.append(handler)

    def process(self, request, *args, **kwargs):
        res = request.get_response(self.application)
        # currently request handlers are un-ordered
        for handler in self.handlers:
            response = handler(request, res)
        return response

在正常的处理流程之后,追加执行了增加的handler处理。
回到一开始,获取到这个请求扩展的字典后,依次遍历,对应的controller增加扩展中指定的handler。

这样三种类型的扩展就完成了。再往后:

        self._router = routes.middleware.RoutesMiddleware(self._dispatch,
                                                          mapper)

又见到了熟悉的RoutesMiddleware,这里就不再累述了,可以参照:https://blog.try-except.com/technology/neutron-server-api.html

上面就是核心资源以外的扩展资源的路由配置。

扩展这一块,其实还少了一部分内容。OpenStack其实还允许对核心资源进行扩展,那么这部分代码在哪里实现的呢?
neutron.api.v2.router:APIRouter

class APIRouter(base_wsgi.Router):

    ...略...

    def __init__(self, **local_config):
        mapper = routes_mapper.Mapper()
        manager.init()
        plugin = directory.get_plugin()
        ext_mgr = extensions.PluginAwareExtensionManager.get_instance()
        ext_mgr.extend_resources("2.0", attributes.RESOURCE_ATTRIBUTE_MAP)
        ...略...

上面的

ext_mgr.extend_resources("2.0", attributes.RESOURCE_ATTRIBUTE_MAP)

这部分内容就实现了这个功能。

发表新评论

© 2017 Powered by Typecho
苏ICP备15035969号-3