I used to work with IBM's Natural Language Understanding API for analyzing URLs. I am using Python's IBM Watson SDK 5.1 on Python 3.8.
I successfully used the code below [all approprioate options have been imported] to extract metadata, in addition to entities, concepts, etc:
def NLU_analysis(url):
try:
response = natural_language_understanding.analyze(
url=url, return_analyzed_text=True, clean=True, language=True,
features=Features(keywords=KeywordsOptions(limit=10),
entities=EntitiesOptions(limit=10),
concepts=ConceptsOptions(limit=5),
metadata=MetadataOptions(),
categories=CategoriesOptions(limit=5))).get_result()
return response
except:
pass
The code above used to return the metadata. Now, in Python SDK 5.1.0, IBM recently changed to way to retrieve the URL's metadata. The "MetadataOptions" feature has been replaced by "FeatureMetadataResults". If I use the code above and replace the MetadataOptions by FeatureMetadataResults as shown below:
def NLU_analysis(url):
try:
response = natural_language_understanding.analyze(
url=url, return_analyzed_text=True, clean=True, language=True,
features=Features(keywords=KeywordsOptions(limit=10),
entities=EntitiesOptions(limit=10),
concepts=ConceptsOptions(limit=5),
metadata=FeaturesResultsMetadata(),
categories=CategoriesOptions(limit=5))).get_result()
return response
except:
pass
Now, if I run the modified code, I get the following error message:
"TypeError: Object of type FeaturesResultsMetadata is not JSON serializable"
If I read IBM's documentation, I am getting somewhat confused (Link to the API documentation. Here's the IBM's code example:
import json
from ibm_watson import NaturalLanguageUnderstandingV1
from ibm_cloud_sdk_core.authenticators import IAMAuthenticator
from ibm_watson.natural_language_understanding_v1
import Features, MetadataOptions
authenticator = IAMAuthenticator('{apikey}')
natural_language_understanding = NaturalLanguageUnderstandingV1(
version='2020-08-01',
authenticator=authenticator
)
natural_language_understanding.set_service_url('{url}')
response = natural_language_understanding.analyze(
url='www.ibm.com',
features=Features(metadata=MetadataOptions())).get_result()
Does anyone know whether it is still possible to retrieve an URL's metadata using IBM Watson's Natural Language Understanding API?
Have a nice day!
It appears that the sample in IBM's API-documentation is incorrect.
The code below has been pasted as plain text, to be able to strike through obsolete elements in IBM's sample code.
import json
from ibm_watson import NaturalLanguageUnderstandingV1
from ibm_cloud_sdk_core.authenticators import IAMAuthenticator
from ibm_watson.natural_language_understanding_v1
import Features, MetadataOptions
authenticator = IAMAuthenticator('{apikey}')
natural_language_understanding = NaturalLanguageUnderstandingV1(
version='2020-08-01',
authenticator=authenticator
)
natural_language_understanding.set_service_url('{url}')
response = natural_language_understanding.analyze(
url='www.ibm.com',
features=Features(metadata=MetadataOptions() {} )).get_result()
print(json.dumps(response, indent=2))
So for requesting the metadata object, just provide an empty dictionary (metadata={})
Enjoy your day!
Joost
Related
I was Googling around to understand how boto3 paginator works, and found a solution that potentially doesn't require writing any logic with NextToken and While loops.
Still, I'm not quite sure what I'm getting when I'm using this:
client = boto3.client('ec2', region_name='eu-west-1')
results = (
client.get_paginator('describe_instances')
.paginate()
.build_full_result()
)
print(results)
I got a huge JSON output and I'm not sure whether I got what I wanted, which is basically the output of all of my EC2 instances.
I'm also not sure how to loop over it, I keep getting TypeError: string indices must be integers which didn't happen before when I used something like:
for instance in response_iterator:
instance = instance['Reservations'][0]
instance_id = instance['Instances'][0]['InstanceId']
print(instance_id)
I would love to understand how to use the build_full_result() method.
I saw a post that says that it's not documented yet, pretty recent to now (as of writing this post).
Interesting find.. this isn't mentioned anywhere in the latest version of boto3 documentation, however it does appear to properly return all available results.
Below is an example from Lambda that shows how to perform a simple loop through the response.. you can update the last two lines to handle the response syntax from EC2 describe instances.
import boto3
client = boto3.client('lambda')
results = (
client.get_paginator('list_functions')
.paginate()
.build_full_result()
)
for result in results['Functions']:
print(result['FunctionName'])
I'm writing an application based on GCP services and I need to access to an external project. I stored on my Firestore database the authentication file's informations of the other project I need to access to. I read this documentation and I tried to apply it but my code does not work. As the documentaion says, what I pass to the authentication method is a dictionary[str, str].
This is my code:
from googleapiclient import discovery
from google.oauth2 import service_account
from google.cloud import firestore
project_id = body['project_id']
user = body['user']
snap_id = body['snapshot_id']
debuggee_id = body['debuggee_id']
db = firestore.Client()
ref = db.collection(u'users').document(user).collection(u'projects').document(project_id)
if ref.get().exists:
service_account_info = ref.get().to_dict()
else:
return None, 411
credentials = service_account.Credentials.from_service_account_info(
service_account_info,
scopes=['https://www.googleapis.com/auth/cloud-platform'])
service = discovery.build('clouddebugger', 'v2', credentials=credentials)
body is just a dictionary containing all the informations of the other project. What I can't understand is why this doesn't work and instead using the method from_service_account_file it works.
The following code will give to that method the same informations of the previous code, but inside a json file instead of a dictionary. Maybe the order of the elements is different, but I think that it doesn't matter at all.
credentials = service_account.Credentials.from_service_account_file(
[PATH_TO_PROJECT_KEY_FILE],
scopes=['https://www.googleapis.com/auth/cloud-platform'])
Can you tell me what I'm doing wrong with the method from_service_account_info?
Problem solved. When I posted the question I manually inserted from the GCP Firestore Console all the info about the other project. Then I wrote the code to make it authomatically and it worked. Honestly I don't know why it didn't worked before, the informations put inside Firestore were the same and the format as well.
I'm trying to use the Google Cloud Platform Natural Language API with Python within a Google Cloud Function. Whenever I use the code provided in this Google tutorial for analyzing entity analysis using text in Cloud Storage, I get the following error message:
File "/user_code/main.py", line 9, in entity_sentiment_file
type=enums.Document.Type.PLAIN_TEXT)
TypeError: <Request 'http://25e4801f1004e4eb41d11633d9b2e9e9-dot-ad6bdc7c397c15e62-tp.appspot.com/'
[POST]> has type LocalProxy, but expected one of: bytes, unicode
I obtain that error message after successfully deploying the function and clicking "Test the Function" with a triggering event of empty curly braces {}, then going to the View Logs page.
I've tried providing the test event parameters like below, but I obtained the same result.
{"gcs_uri":"gs://test-news-articles/news-article-1.txt"}
Here's my entire function:
from google.cloud import language
from google.cloud.language import enums
from google.cloud.language import types
def entity_sentiment_file(gcs_uri,request=None):
print('gcs_uri: {}'.format(gcs_uri))
client = language.LanguageServiceClient()
document = types.Document(
gcs_content_uri=gcs_uri,
type=enums.Document.Type.PLAIN_TEXT)
# Detect and send native Python encoding to receive correct word offsets.
encoding = enums.EncodingType.UTF32
if sys.maxunicode == 65535:
encoding = enums.EncodingType.UTF16
result = client.analyze_entity_sentiment(document, encoding)
for entity in result.entities:
print(u'Name: "{}"'.format(entity.name))
for mention in entity.mentions:
print(u' Begin Offset : {}'.format(mention.text.begin_offset))
print(u' Content : {}'.format(mention.text.content))
print(u' Magnitude : {}'.format(mention.sentiment.magnitude))
print(u' Sentiment : {}'.format(mention.sentiment.score))
print(u' Type : {}'.format(mention.type))
print(u'Salience: {}'.format(entity.salience))
print(u'Sentiment: {}\n'.format(entity.sentiment))
Any help would be much appreciated.
A function that responds to an HTTP request needs to have the signature:
def my_function(request):
...
where request is provided by the Cloud Functions runtime on every new request.
Right now, gcs_uri is getting set to the request value (which is a LocalProxy type) and then you're trying to format a string with it, which causes the exception.
I'm not sure where you're expecting gcs_uri to come from, but it won't be provided to the function as an argument. If you're making a request with JSON, it will be available using request.json['gcs_uri']. See "Writing HTTP Functions" for more details.
I am trying to combine multiple logic adapters in Python chatterbot. I cannot seem to get it right. I tried this:
english_bot = ChatBot("English Bot",
storage_adapter="chatterbot.storage.SQLStorageAdapter",
multi_logic_adapter = [
"chatterbot.logic.MathematicalEvaluation",
"chatterbot.logic.TimeLogicAdapter",
"chatterbot.logic.BestMatch"]
)
Only BestMatch seems to be active
And I tried this:
english_bot = ChatBot("English Bot",
storage_adapter="chatterbot.storage.SQLStorageAdapter",
logic_adapter = [
"chatterbot.logic.multi_adapter.MultiLogicAdapter",
"chatterbot.logic.MathematicalEvaluation",
"chatterbot.logic.TimeLogicAdapter",
"chatterbot.logic.BestMatch"]
)
But I get this error: AttributeError: 'NoneType' object has no attribute 'confidence' and none of the logic_adapters seems to be active.
Thanks,
Herb
BestMatch
Adapter is the default adapter for chatterbot, You don't need explicitly specify that. More information http://chatterbot.readthedocs.io/en/stable/logic/index.html#best-match-adapter
And you code should like this
# -*- coding: utf-8 -*-
from chatterbot import ChatBot
bot = ChatBot(
"English Bot",
logic_adapters=[
"chatterbot.logic.MathematicalEvaluation",
"chatterbot.logic.TimeLogicAdapter"
]
)
# Print an example of getting one math based response
response = bot.get_response("What is 4 + 9?")
print(response)
# Print an example of getting one time based response
response = bot.get_response("What time is it?")
print(response)
Every logic adapter in logic_adapters=[] is automatically processed by MultiLogicAdapter. You might need to tweak the confidence levels though.
More info about the MultiLogicAdapter here:
http://chatterbot.readthedocs.io/en/stable/logic/multi-logic-adapter.html
Multi Logic adapter is an inbuilt class and is not explicitly defined in the code.You can see this statement in the introduction part:"ChatterBot internally uses a special logic adapter that allows it to choose the best response generated by any number of other logic adapters" This is the link - http://chatterbot.readthedocs.io/en/stable/logic/multi-logic-adapter.html
Also, similar query is already available on stackover flow. Refer this as well.
Error while using chatterbot
The MultiLogicAdapter typically doesn't get used directly in this way.
Each logic adapter that you add to the logic_adapters=[] will get processed by the MultiLogicAdapter internally by ChatterBot, no need to explicitly specify it.
How can I translate the error messages from the colander validators? The documentation just says that it's possible.
def valid_text(node, value):
raise Invalid(node, u"Some error message")
class form(colander.MappingSchema):
name = colander.SchemaNode(colander.String(), validator=valid_text)
I know that deform does it already but I need to use colander on his own.
According to the API documentation, the msg argument to Invalid can be a translation string instance. Information on working with translation strings is here.
Looks like this issue was already addressed and fixed, but it will be part of the next release. I've just added the changes from commit f6be836 and it works like a charm.