Cleaning tweet datasets by removing annoying character in bytecode (exp : \xf0\x9f\x99\x82)
Here's the code without using function :
b = data_tweet['Tweet']
b.head()
for i in b:
x = i.encode('utf=8')
y = x.decode('unicode-escape')
print(y)
It worked. The character became : 🙄, 🥰, etc.
But when I implemented it using function, in order to convert it in csv file. it failed. The byte character stays the same (exp : \xf0\x9f\x99\x82)
Here's the code :
def convert(text):
for i in text:
x = i.encode('utf=8')
y = x.decode('unicode-escape')
return text
convert(data_tweet['Tweet'])
Does anyone know why?
Problem is that you actually didn't assign the result to data_tweet['Tweet']. You can use apply() on Series.
def convert(text):
x = text.encode('utf=8')
y = x.decode('unicode-escape')
return y
data_tweet['Tweet'] = data_tweet['Tweet'].apply(convert)
Or
data_tweet['Tweet'] = data_tweet['Tweet'].apply(lambda text: text.encode('utf=8').decode('unicode-escape'))
Related
Hi I'm starting to play around with NIM
I get a "can't evaluate at compile time" error on this code:
import strutils
type
Matrix[x, y: static[int], T] = object
data: array[x * y, T]
var n,m: int = 0
proc readFile() =
let f = open("matrix.txt")
defer: f.close()
var graph_size = parseInt(f.readline)
var whole_graph: Matrix[graph_size, graph_size, int]
for line in f.lines:
for field in line.splitWhitespace:
var cell = parseInt(field)
whole_graph[n][m] = cell
m = m + 1
n = n + 1
readFile()
Any help appreciated.
Unless you absolutely positively need array in this scenario while not knowing its size at compile-time, you may want to rather swap to the seq type, whose size does not need to be known at compile-time.
Together with std/enumerate you can even save yourself the hassle of tracking the index with n and m:
import std/[strutils, enumerate]
type Matrix[T] = seq[seq[T]]
proc newZeroIntMatrix(x: int, y: int): Matrix[int] =
result = newSeqOfCap[seq[int]](x)
for i in 0..x-1:
result.add(newSeqOfCap[int](y))
for j in 0..y-1:
result[i].add(0)
proc readFile(): Matrix[int] =
let f = open("matrix.txt")
defer: f.close()
let graph_size = parseInt(f.readline)
var whole_graph = newZeroIntMatrix(graph_size, graph_size)
for rowIndex, line in enumerate(f.lines):
for columnIndex, field in enumerate(line.split):
let cell = parseInt(field)
whole_graph[rowIndex][columnIndex] = cell
result = whole_graph
let myMatrix = readFile()
echo myMatrix.repr
Further things I'd like to point out though are:
array[x * y, T] will not give you a 2D array, but a single array of length x*y. If you want a 2D array, you would most likely want to store this as array[x, array[y, T]]. That is assuming that you know x and y at compile-time, so your variable declaration would look roughly like this: var myMatrix: array[4, array[5, int]]
Your Matrix type has the array in its data field, so trying to access the array with that Matrix type needs to be done accordingly (myMatrix.data[n][m]). That is, unless you define proper []and []= procs for the Matrix type that do exactly that under the hood.
I have
import sympy as sm
x = sm.symbols('x', cls=sm.Function)
t = sm.symbols('t')
expr = x(t).diff(t) + 0.05*x(t)
sol = sm.dsolve(expr,x(t), ics = {x(0):25})
Now i have the solution as an relational equality. Now i want to solve t for x = 1. I can't do
s = sm.Eq(x,-1) to do sm.solve(s,t) as s returns False
figured it out. It's simply easy. x.rhs can be used to create an equation to use solve for t.
equation = sm.Eq(sol.rhs,1)
sm.solve(equation,t)
gives the result t ~ 64.38
From this
import sympy as sp
x,y,z = sp.symbols("x y z")
sp.Ep(x,y/z)
To this
#varibles = array
#equation = ????
def solver(variables,equation):
#Looping through variables array and converting variables to sympy objects
for var in variables:
var = sp.symbols(var)
#Generate sympy Equation
equation = sp.Ep(equation)
variables = [x,y,z]
equation = x,y/z #invalid code
solver(variables,equation)
I'm creating a function that is able to take in an equation with x amount of variables and x-1 number of values then solve for the missing variable symbolically then return a numerical answer using the values provided.
I only included a small portion of code where I'm having trouble understanding how to pass through an equation. Any solutions or pointers would be greatly appericated. Thanks.
There are several layers of potential confusion here concerning Python variables and SymPy objects (Symbols) used for variables.
Here is an example of what you are saying:
# 3 variables
syms = x, y, z = var('x:z')
# 2 values
vals = {x:1, y:2}
# an equations
eq = Eq(x, y/z)
# solve for the missing value symbolically
missing = set(syms) - set(vals) # == {z}
solve(eq, missing)
[y/x]
# solve for the missing value after substituting in the known values
solve(eq.subs(vals))
[2]
You could make a solver to accept an equation and then specified values and figure out the missing one and return that value by doing something like this:
>>> def solver(eq, **vals):
... from sympy.core.containers import Dict
... from sympy.solvers.solvers import solve
... free = eq.free_symbols
... vals = Dict(vals)
... x = free - set(vals)
... if len(x) != 1:
... raise ValueError('specify all but one of the values for %s' % free)
... x = x.pop()
... return solve(eq.subs(vals), x, dict=True)
...
>>> solver(eq, x=1, z=2)
[{y: 2}]
Does that give you some ideas of how to continue?
I am trying to use sympy to solve a function symbolically, then input the values in and solve it numerically. I can do this with just one variable, but can't figure out how to do it with multiple. Here is what I have so far.
v,v0,a,t = sp.var('v v0 a t')
args = [v0,a,t]
arg_vals = [1,-9.81,2]
def get_function():
v = v0 + a*t
return v
def get_derivative(fun,var):
derivative = sp.diff(fun,var)
return derivative
def get_integral(fun,var):
integral = sp.integrate(fun,var)
return integral
def eval_function(fun, args, arg_vals):
i=0
for i in range(len(arg_vals)):
args[i] = arg_vals[i]
return fun.evalf(subs={args})
v = get_function()
a = get_derivative(v,t)
x = get_integral(v,t)
x_eval = eval_function(v,args,arg_vals)
The code runs fine until it hits the eval_function return fun.evalf(subs={args}). Then I get
>>>TypeError: unhashable type: 'list'
I've been trying to figure this out for a while, and guess that someone can just take a few seconds and tell me I'm dumb lol. Thanks for the help!!
(using anaconda, spyder, python 3)
i am scraping multiple websites so i am using one function for each website script, so each function returns 4 values, i want to print them in dataframe and write them in csv but i am facing this problem, i may be asking something too odd or basic but please help
Either i will have to write whole script in one block and that will look very nasty to handle so if i could find a way around, this is just a sample of problem i am facing..
def a1(x):
z=x+1
r = x+2
print(z, r)
def a2(x):
y=x+4
t=x+3
print(y, t)
x = 2
a1(x)
a2(x)
3 4
6 5
data = pd.Dataframe({'first' : [z],
'second' : [r],
'third' : [y],
'fourth' : [t]
})`
data
*error 'z' is not defined*
You may find it convenient to write functions that return a list of dicts.
For example:
rows = [dict(a=1, b=2, c=3),
dict(a=4, b=5, c=6)]
df = pd.DataFrame(rows)
The variables are only defined in the local scope of your functions, you'd either need to declare them globally or - the better way - return them so you can use them outside of the function by assigning the return values to new variables
import pandas as pd
def a1(x):
z = x+1
r = x+2
return (z, r)
def a2(x):
y = x+4
t = x+3
return (y, t)
x = 2
z, r = a1(x)
y, t = a2(x)
data = pd.DataFrame({'first' : [z],
'second' : [r],
'third' : [y],
'fourth' : [t]
})