neutron除了核心资源以外还有扩展资源,核心资源的API处理实现流程已经分析过了(见:https://blog.try-except.com/technology/neutron-server-api.html)。
那么neutron是如何支持扩展资源的API并进行相关的路由配置呢?
其实答案也在paste-api.conf文件中,其中有这样一段配置:
[filter:extensions]
paste.filter_factory = neutron.api.extensions:plugin_aware_extension_middleware_factory
这个extensions的filter在neutron-server启动的时候会被加载,先看一下它的构造方法:
def plugin_aware_extension_middleware_factory(global_config, **local_config):
"""Paste factory."""
def _factory(app):
ext_mgr = PluginAwareExtensionManager.get_instance()
return ExtensionMiddleware(app, ext_mgr=ext_mgr)
return _factory
构造了一个PluginAwareExtensionManager
对象,并用它继续构造了一个ExtensionMiddleware
对象。对于PluginAwareExtensionManager
,我们有必要看一下它的创建过程get_instance()
:
@classmethod
def get_instance(cls):
if cls._instance is None:
service_plugins = directory.get_plugins()
cls._instance = cls(get_extensions_path(service_plugins),
service_plugins)
return cls._instance
很容易看出来这其实是一个单例模式的构造过程。
service_plugins = directory.get_plugins()
这一句是获取到了所有已经加载了的service plugin对象,至于service plugin对象是怎么加载的,我会另写一篇文章介绍,这里避免内容过多就不再叙说。
get_extensions_path(service_plugins)
这一句也是一个关键,根据加载的service plugin获取extension的搜索路径,为什么获取extensino的存放路径要用到service plugin呢?
def get_extensions_path(service_plugins=None):
paths = collections.OrderedDict()
# Add Neutron core extensions
paths[core_extensions.__path__[0]] = 1
if service_plugins:
# Add Neutron *-aas extensions
for plugin in service_plugins.values():
neutron_mod = provider_configuration.NeutronModule(
plugin.__module__.split('.')[0])
try:
paths[neutron_mod.module().extensions.__path__[0]] = 1
except AttributeError:
# Occurs normally if module has no extensions sub-module
pass
# Add external/other plugins extensions
if cfg.CONF.api_extensions_path:
for path in cfg.CONF.api_extensions_path.split(":"):
paths[path] = 1
LOG.debug("get_extension_paths = %s", paths)
# Re-build the extension string
path = ':'.join(paths)
return path
首先创建了一个OrderedDict
,也就是一个有序的字典对象paths
。
paths[core_extensions.__path__[0]] = 1
这句话向paths
字典添加了核心资源的扩展路径,core_extensions
就是neutron.extensions
模块
from neutron import extensions as core_extensions
所以core_extensions.__path__[0]
就是neutron.extensions
的真实路径。接下来遍历的所有的service_plugins。
neutron_mod = provider_configuration.NeutronModule(
plugin.__module__.split('.')[0])
上面这句话中plugin.__module__.split('.')[0]
这句话的意思是取plugin模块名中第一个点之前的名称。举个例子:如果一个service plugin的代码实现是在neutron这个大模块下面的,例如以下的service plugin:
[neutron.service_plugins]
auto_allocate = neutron.services.auto_allocate.plugin:Plugin
dummy = neutron.tests.unit.dummy_plugin:DummyServicePlugin
flavors = neutron.services.flavors.flavors_plugin:FlavorsPlugin
loki = neutron.services.loki.loki_plugin:LokiPlugin
metering = neutron.services.metering.metering_plugin:MeteringPlugin
qos = neutron.services.qos.qos_plugin:QoSPlugin
revisions = neutron.services.revisions.revision_plugin:RevisionPlugin
router = neutron.services.l3_router.l3_router_plugin:L3RouterPlugin
segments = neutron.services.segments.plugin:Plugin
tag = neutron.services.tag.tag_plugin:TagPlugin
timestamp = neutron.services.timestamp.timestamp_plugin:TimeStampPlugin
trunk = neutron.services.trunk.plugin:TrunkPlugin
它们的模块名称第一个点之前的都是neutron
,那么plugin.__module__.split('.')[0]
的值就是neutron。
而provider_configuration.NeutronModule
对象的module()
返回的就是构造时传入的module名对应的module对象。
try:
paths[neutron_mod.module().extensions.__path__[0]] = 1
except AttributeError:
# Occurs normally if module has no extensions sub-module
pass
所以neutron_mod.module().extensions.__path__[0]
指的就是service plugin顶层模块下面的extensions模块的路径,例如:neutron.extensions
模块的路径。
那么同理,像下面所列的service plugin:
[neutron.service_plugins]
firewall = neutron_fwaas.services.firewall.fwaas_plugin:FirewallPlugin
firewall_v2 = neutron_fwaas.services.firewall.fwaas_plugin_v2:FirewallPluginV2
它们的extensions路径就是模块neutron_fwaas.extensinos
对应的路径了。
这里就明白了,为啥获取extension的搜索路径要通过service_plugin,就是需要通过service plugin的模块名来计算对应的path。
除此以外,用户也可以在配置文件中指定其他的路径地址,也会被加到paths
字典中。
if cfg.CONF.api_extensions_path:
for path in cfg.CONF.api_extensions_path.split(":"):
paths[path] = 1
最后所有的路径会用冒号连接并返回。
path = ':'.join(paths)
return path
类似CLASSPATH一样。
继续往下:
cls._instance = cls(get_extensions_path(service_plugins),
service_plugins)
返回的路径和所有的service plugin对象再被用来创建extension manager:PluginAwareExtensionManager对象。
接下来就来看一下ExtensionMiddleware
对象的创建,其__init__
方法如下:
def __init__(self, application,
ext_mgr=None):
self.ext_mgr = (ext_mgr
or ExtensionManager(get_extensions_path()))
mapper = routes.Mapper()
# extended resources
for resource in self.ext_mgr.get_resources():
path_prefix = resource.path_prefix
if resource.parent:
path_prefix = (resource.path_prefix +
"/%s/{%s_id}" %
(resource.parent["collection_name"],
resource.parent["member_name"]))
LOG.debug('Extended resource: %s',
resource.collection)
for action, method in six.iteritems(resource.collection_actions):
conditions = dict(method=[method])
path = "/%s/%s" % (resource.collection, action)
with mapper.submapper(controller=resource.controller,
action=action,
path_prefix=path_prefix,
conditions=conditions) as submap:
submap.connect(path_prefix + path, path)
submap.connect(path_prefix + path + "_format",
"%s.:(format)" % path)
for action, method in resource.collection_methods.items():
conditions = dict(method=[method])
path = "/%s" % resource.collection
with mapper.submapper(controller=resource.controller,
action=action,
path_prefix=path_prefix,
conditions=conditions) as submap:
submap.connect(path_prefix + path, path)
submap.connect(path_prefix + path + "_format",
"%s.:(format)" % path)
mapper.resource(resource.collection, resource.collection,
controller=resource.controller,
member=resource.member_actions,
parent_resource=resource.parent,
path_prefix=path_prefix)
# extended actions
action_controllers = self._action_ext_controllers(application,
self.ext_mgr, mapper)
for action in self.ext_mgr.get_actions():
LOG.debug('Extended action: %s', action.action_name)
controller = action_controllers[action.collection]
controller.add_action(action.action_name, action.handler)
# extended requests
req_controllers = self._request_ext_controllers(application,
self.ext_mgr, mapper)
for request_ext in self.ext_mgr.get_request_extensions():
LOG.debug('Extended request: %s', request_ext.key)
controller = req_controllers[request_ext.key]
controller.add_handler(request_ext.handler)
self._router = routes.middleware.RoutesMiddleware(self._dispatch,
mapper)
super(ExtensionMiddleware, self).__init__(application)
一开始就见到了熟悉的mapper对象,所以可以确定路由设定肯定是在这里。
参数application就是用来创建middleware的WSGI app。
参数ext_mgr是上面已经创建好的PluginAwareExtensionManager对象。
现在我们需要知道extensino扩展有三种情况
相关代码:
# extended resources
for resource in self.ext_mgr.get_resources():
path_prefix = resource.path_prefix
if resource.parent:
path_prefix = (resource.path_prefix +
"/%s/{%s_id}" %
(resource.parent["collection_name"],
resource.parent["member_name"]))
LOG.debug('Extended resource: %s',
resource.collection)
for action, method in six.iteritems(resource.collection_actions):
conditions = dict(method=[method])
path = "/%s/%s" % (resource.collection, action)
with mapper.submapper(controller=resource.controller,
action=action,
path_prefix=path_prefix,
conditions=conditions) as submap:
submap.connect(path_prefix + path, path)
submap.connect(path_prefix + path + "_format",
"%s.:(format)" % path)
for action, method in resource.collection_methods.items():
conditions = dict(method=[method])
path = "/%s" % resource.collection
with mapper.submapper(controller=resource.controller,
action=action,
path_prefix=path_prefix,
conditions=conditions) as submap:
submap.connect(path_prefix + path, path)
submap.connect(path_prefix + path + "_format",
"%s.:(format)" % path)
mapper.resource(resource.collection, resource.collection,
controller=resource.controller,
member=resource.member_actions,
parent_resource=resource.parent,
path_prefix=path_prefix)
与核心资源不同,核心资源的资源是固定的,所以neutron通过一个字典数据结构将这些资源罗列了出来:
RESOURCES = {'network': 'networks',
'subnet': 'subnets',
'subnetpool': 'subnetpools',
'port': 'ports'}
而扩展资源由于用户可以自定义扩展,无法提前预知,所以需要动态的获取:
for resource in self.ext_mgr.get_resources():
这里调用了ext_mgr的get_resources
方法来获取的:
def get_resources(self):
"""Returns a list of ResourceExtension objects."""
resources = []
resources.append(ResourceExtension('extensions',
ExtensionController(self)))
for ext in self.extensions.values():
resources.extend(ext.get_resources())
return resources
resources列表先追加了一个ResourceExtension类型的资源,这就是扩展了一个资源,它的collection名就是extensions。相当与实现了/v2.0/extensions。
剩下的资源获取都是通过遍历了ext_mgr
中的所有的extension,调用了每个extension对象的get_resources
方法。所以需要先看一下extension是怎么被加载到extension manager中的。主要方法是其中的_load_all_extensions
且是在extension manager初始化的时候被加载的:
def __init__(self, path):
LOG.info(_LI('Initializing extension manager.'))
self.path = path
self.extensions = {}
self._load_all_extensions()
这里的path就是前面所讲的extension模块的路径集合,路径和路径之间用冒号连接。加载extension是调用了_load_all_extensions
方法:
def _load_all_extensions(self):
for path in self.path.split(':'):
if os.path.exists(path):
self._load_all_extensions_from_path(path)
else:
LOG.error(_LE("Extension path '%s' doesn't exist!"), path)
遍历了所有的extension路径,如果路径存在,则对这些路径调用_load_all_extensions_from_path
方法:
def _load_all_extensions_from_path(self, path):
for f in sorted(os.listdir(path)):
try:
LOG.debug('Loading extension file: %s', f)
mod_name, file_ext = os.path.splitext(os.path.split(f)[-1])
ext_path = os.path.join(path, f)
if file_ext.lower() == '.py' and not mod_name.startswith('_'):
mod = imp.load_source(mod_name, ext_path)
ext_name = mod_name[0].upper() + mod_name[1:]
new_ext_class = getattr(mod, ext_name, None)
if not new_ext_class:
LOG.warning(_LW('Did not find expected name '
'"%(ext_name)s" in %(file)s'),
{'ext_name': ext_name,
'file': ext_path})
continue
new_ext = new_ext_class()
self.add_extension(new_ext)
except Exception as exception:
LOG.warning(_LW("Extension file %(f)s wasn't loaded due to "
"%(exception)s"),
{'f': f, 'exception': exception})
在这个方法中遍历了指定路径下的所有文件,当文件名的后缀是".py"并且文件名不以"_
"开头,就会去加载这个py文件。加载完成后会去find与文件名同名(首字母大写)的类,这个类就是需要加载的extension类。如果找到了就会创建这个类的对象,并通过add_extension
方法将这个extension对象保存到manager对像中。
这样就完成了所有extension的加载。
回到get_resources
方法:
def get_resources(self):
"""Returns a list of ResourceExtension objects."""
resources = []
resources.append(ResourceExtension('extensions',
ExtensionController(self)))
for ext in self.extensions.values():
resources.extend(ext.get_resources())
return resources
调用完每个extension对象的get_resource方法后把结果保存到列表中返回,这个列表中的每个对象都应该是ResourceExtension类型的。
后面的操作就是遍历每一个resource对象,利用mapper给每个resource做路由设定。
所以,扩展新的资源,需要在对应extension的get_resources方法中返回一个ResourceExtension对象。
相关代码如下:
# extended actions
action_controllers = self._action_ext_controllers(application,
self.ext_mgr, mapper)
for action in self.ext_mgr.get_actions():
LOG.debug('Extended action: %s', action.action_name)
controller = action_controllers[action.collection]
controller.add_action(action.action_name, action.handler)
首先通过_action_ext_controllers
创建了一个collection和controller对应关系的字典:
def _action_ext_controllers(self, application, ext_mgr, mapper):
"""Return a dict of ActionExtensionController-s by collection."""
action_controllers = {}
for action in ext_mgr.get_actions():
if action.collection not in action_controllers.keys():
controller = ActionExtensionController(application)
mapper.connect("/%s/:(id)/action.:(format)" %
action.collection,
action='action',
controller=controller,
conditions=dict(method=['POST']))
mapper.connect("/%s/:(id)/action" % action.collection,
action='action',
controller=controller,
conditions=dict(method=['POST']))
action_controllers[action.collection] = controller
return action_controllers
通过extension manager的get_actions方法获取到所有扩展的action。
def get_actions(self):
"""Returns a list of ActionExtension objects."""
actions = []
for ext in self.extensions.values():
actions.extend(ext.get_actions())
return actions
类似的,也是依次调用了所有的extension的get_actions方法,并将所有结果组成一个list返回。
然后遍历每个action,为每个action都创建了一个controller:ActionExtensionController
class ActionExtensionController(wsgi.Controller):
def __init__(self, application):
self.application = application
self.action_handlers = {}
def add_action(self, action_name, handler):
self.action_handlers[action_name] = handler
def action(self, request, id):
input_dict = self._deserialize(request.body,
request.get_content_type())
for action_name, handler in six.iteritems(self.action_handlers):
if action_name in input_dict:
return handler(input_dict, request, id)
# no action handler found (bump to downstream application)
response = self.application
return response
可以看到每个action name对应一个handler,处理的时候会根据请求调用对应的handler。
之后为每个action添加了两条路由:
mapper.connect("/%s/:(id)/action.:(format)" %
action.collection,
action='action',
controller=controller,
conditions=dict(method=['POST']))
mapper.connect("/%s/:(id)/action" % action.collection,
action='action',
controller=controller,
conditions=dict(method=['POST']))
controller指定为上面创建的controller对象,action方法名定义为"action
"。这样,当请求过来的时候,根据这两条路由,就能执行到上述controller的action方法了。
当所有的action保存到字典后,返回。返回之后遍历所有的action,对每个action执行以下代码:
controller = action_controllers[action.collection]
controller.add_action(action.action_name, action.handler)
为action对应的controller添加需要扩展的方法名和handler,这样就完成了扩展。
扩展的代码如下:
req_controllers = self._request_ext_controllers(application,
self.ext_mgr, mapper)
for request_ext in self.ext_mgr.get_request_extensions():
LOG.debug('Extended request: %s', request_ext.key)
controller = req_controllers[request_ext.key]
controller.add_handler(request_ext.handler)
流程和action扩展十分类似,先通过_request_ext_controllers
获取到一个request扩展的key和controller对应关系的字典。
def _request_ext_controllers(self, application, ext_mgr, mapper):
"""Returns a dict of RequestExtensionController-s by collection."""
request_ext_controllers = {}
for req_ext in ext_mgr.get_request_extensions():
if req_ext.key not in request_ext_controllers.keys():
controller = RequestExtensionController(application)
mapper.connect(req_ext.url_route + '.:(format)',
action='process',
controller=controller,
conditions=req_ext.conditions)
mapper.connect(req_ext.url_route,
action='process',
controller=controller,
conditions=req_ext.conditions)
request_ext_controllers[req_ext.key] = controller
return request_ext_controllers
对每个request extension都创建了一个RequestExtensionController对象,然后通过mapper进行路由的绑定。看一下这个controller:
class RequestExtensionController(wsgi.Controller):
def __init__(self, application):
self.application = application
self.handlers = []
def add_handler(self, handler):
self.handlers.append(handler)
def process(self, request, *args, **kwargs):
res = request.get_response(self.application)
# currently request handlers are un-ordered
for handler in self.handlers:
response = handler(request, res)
return response
在正常的处理流程之后,追加执行了增加的handler处理。
回到一开始,获取到这个请求扩展的字典后,依次遍历,对应的controller增加扩展中指定的handler。
这样三种类型的扩展就完成了。再往后:
self._router = routes.middleware.RoutesMiddleware(self._dispatch,
mapper)
又见到了熟悉的RoutesMiddleware,这里就不再累述了,可以参照:https://blog.try-except.com/technology/neutron-server-api.html
上面就是核心资源以外的扩展资源的路由配置。
扩展这一块,其实还少了一部分内容。OpenStack其实还允许对核心资源进行扩展,那么这部分代码在哪里实现的呢?
neutron.api.v2.router:APIRouter
class APIRouter(base_wsgi.Router):
...略...
def __init__(self, **local_config):
mapper = routes_mapper.Mapper()
manager.init()
plugin = directory.get_plugin()
ext_mgr = extensions.PluginAwareExtensionManager.get_instance()
ext_mgr.extend_resources("2.0", attributes.RESOURCE_ATTRIBUTE_MAP)
...略...
上面的
ext_mgr.extend_resources("2.0", attributes.RESOURCE_ATTRIBUTE_MAP)
这部分内容就实现了这个功能。