Python - How do I....

[Luca Brazzi]

Learning about python for a data analytics qual.
Complete numpty at the moment.

One of my pieces of work is to create a word cloud from a .docx file using Python. Easy I'm sure if you're familiar with Python. Impossible it seems if not.

My understanding is that I need to get all the words in the Word document into a data frame. The document contents are a written text report (formatted in Word). It's free text rather than in tables.

I've used docx2txt to extract the data from the .docx file. This works fine.
text = docx2txt.process("bob.docx")

How do I then put this into a data frame?
Tried:
df = pd.DataFrame(text)

Error message: ValueError: DataFrame constructor not properly called!

I guess the formatting is not right, but don't know how to correct it.

Once I have a working data frame, I think I can manage the word cloud aspect from there.

Any ideas?

Or even starting from scratch?

[eharding]

Try

import pandas as pd
import docx2txt

df_words = pd.DataFrame(docx2txt.process("bob.docx").split(' '))

i.e. split your single string of text into a list of words. You might want to elaborate on the initial processing to remove punctuation, force everything to lowercase and so on.

[Luca Brazzi]

eharding said:

Try

import pandas as pd
import docx2txt

df_words = pd.DataFrame(docx2txt.process("bob.docx").split(' '))

i.e. split your single string of text into a list of words. You might want to elaborate on the initial processing to remove punctuation, force everything to lowercase and so on.

Just to follow up on this.....thank you so much for your help. It worked!
I got the wordcloud working fine....but I cannot seem to be able to use my own mask rather then the default rectangle. I've searched google and found an example using Alice in Wonderland shape as the mask, and saw it was a PNG file....so I tried creating my own PNG but without success.

Do you know of the formatting rules required for making it possible to use a home made mask file for the word cloud shape?

Thanks again,
LB

[-Pete-]

Please can someone help setup Python on a mac for my daughter?

I've followed a recommendation to use a virtual environment so I created one and Python 3.9.5, activated, I've run python3 -m pip install requests, and used cd to get into the environment directory. ls shows bin, include, pyvenv.cfg and lib as subdirectories. Command line mode works fine. We copied the first example from our python book, but it fails with this error:

from random import randint
^
SyntaxError: invalid syntax

Typing "which Python3" gives python3 in the correct virtual environment bin directory. I can't work out how to make it aware of the standard libraries/modules. Can anyone help, I need the easy way to fix this before she loses patience? Or is there's a Python forum which can explain things to a beginner - they all seem to expect prior knowledge, or add extra layers of complexity?

[rustyuk]

I have Python on my Mac and have never used a Virtual Environment.

If you are just learning there is no need to install anything just use one of the many online shells.

[frisbee]

I've only got Python 2.7 installed on my PC at home but your example import worked for me.

I've not used virtual envs before as they are incredibly slow on my work laptop.

You could just run it in the IDE that comes with Python? IDLE

[xeny]

frisbee said:

I
I've not used virtual envs before as they are incredibly slow on my work laptop.

Are you thinking of Virtual Machines? Virtual Environments should be full speed - it's just a way of managing multiple python installs in the same OS installation - https://docs.python.org/3/tutorial/venv.html

[-Pete-]

Thanks for the replies. Mac's come with Python 2.7 and Python 3 installed these days, but apparently Apple use Python as part of the OS so there's a risk changing config settings. But I have tried Python 2.7 and Python 3 before I setup the 'safe' virtual environment, and had the same problem.

Why can't my python find standard modules such as random?

[frisbee]

xeny said:

Are you thinking of Virtual Machines? Virtual Environments should be full speed - it's just a way of managing multiple python installs in the same OS installation - https://docs.python.org/3/tutorial/venv.html

The initial setup of a Virtualenv project on Pycharm is incredibly slow. What's almost instantaneous on my home computer, is "wander off make a coffee and chat to a few people" slow on my work computer. The oodles of corporate spyware probably!

[frisbee]

Is that the first line in the program?

An error could be carrying over from a previous line, if you hadn't closed a bracket for example.

[-Pete-]

Thanks for the suggestion, but no, the first line is hash Ghost Game (PH does some odd formatting if I type hash) followed by:
from random import randint
print("Ghost Game")
etc

If I comment out the 2nd line, it runs fine until it gets to the randint() call.

The example is from the book "Computer Programming" published by DK and they don't mention setup, they just say run the installer, open IDLE, type and save the program and run it. I've been programming since the 1970's so the code isn't the problem, just the setup

Edited by -Pete- on Sunday 6th June 21:42

[-Pete-]

I think you're onto something, I'm getting random numbers now.
Many many thanks!

[Luca Brazzi]

Luca Brazzi said:

Just to follow up on this.....thank you so much for your help. It worked!
I got the wordcloud working fine....but I cannot seem to be able to use my own mask rather then the default rectangle. I've searched google and found an example using Alice in Wonderland shape as the mask, and saw it was a PNG file....so I tried creating my own PNG but without success.

Do you know of the formatting rules required for making it possible to use a home made mask file for the word cloud shape?

Thanks again,
LB

Happy for the thread-diversion to bring this back on to the forum….but to go back to my question….does anyone have any ideas on the mask formatting?
Ta

[eharding]

Luca Brazzi said:

Luca Brazzi said:

Just to follow up on this.....thank you so much for your help. It worked!
I got the wordcloud working fine....but I cannot seem to be able to use my own mask rather then the default rectangle. I've searched google and found an example using Alice in Wonderland shape as the mask, and saw it was a PNG file....so I tried creating my own PNG but without success.

Do you know of the formatting rules required for making it possible to use a home made mask file for the word cloud shape?

Thanks again,
LB

Happy for the thread-diversion to bring this back on to the forum….but to go back to my question….does anyone have any ideas on the mask formatting?
Ta

Need some indication of what packages you are using for the image generation - Pillow? Easy enough to deduce that you’re using Pandas from the first code snippet, but difficult to give advice on the second part without more information.

[NowWatchThisDrive]

Luca Brazzi said:

Happy for the thread-diversion to bring this back on to the forum….but to go back to my question….does anyone have any ideas on the mask formatting?
Ta

Never tried generating a "word cloud" before, but done plenty of Python so happy to try and help

Am I right in thinking this is the example you're referring to - and so you're using the 'wordcloud' module as it prescribes? https://medium.com/analytics-vidhya/data-visualiza...

Do you have the issue regardless of what PNG you use, or just this one? What is the PNG (if you're able to link it)? Even better if you can include a code snippet.

[eharding]

Example done with PIL/Pillow, using requests to grab a font and mask image....

from PIL import Image, ImageDraw, ImageFont
from random import randint
from io import BytesIO
import requests

background_image = Image.new('RGB', (512,512), color = 0xFFFFFF)

text_image = Image.new('RGB', (512,512), color = 0x000000)

text_image_draw = ImageDraw.Draw(text_image)

word_array = "Powerfully Built Company Directors".split(' ')

font_get_response = requests.get("https://tinyurl.com/4rr7uaed")

ttf_font = ImageFont.truetype(BytesIO(font_get_response.content), size=15)

for _ in range(0,200):
  the_word =  word_array[randint(0, len(word_array) - 1)]
  text_image_draw.text((randint(0,511), randint(0,511)), the_word, fill = 0x00FF00, font = ttf_font)

mask_logo_get_response = requests.get('https://thumbsnap.com/i/2g7fS7iP.png')

mask_logo_image = Image.open(BytesIO(logo_get_response.content))

composite_image = Image.composite(background_image, text_image, mask_logo_image)

Edited: thumbsnap.com wierdness prevented the result being shown....in your further example, just render some text in a large font into your mask image instead.

Edited by eharding on Monday 7th June 16:28

[Luca Brazzi]

NowWatchThisDrive said:

Never tried generating a "word cloud" before, but done plenty of Python so happy to try and help

Am I right in thinking this is the example you're referring to - and so you're using the 'wordcloud' module as it prescribes? https://medium.com/analytics-vidhya/data-visualiza...

Do you have the issue regardless of what PNG you use, or just this one? What is the PNG (if you're able to link it)? Even better if you can include a code snippet.

Thanks for offering to help....complete numpty at python as I'm not a programmer.
Your link is EXACTLY the think I'm trying to do.

I don't have a mask at the moment....I'm wanting to create a text mask....for example big letters spellings out 'ACTIONS' filled with the word cloud results.
I just cannot get it to work.

[Luca Brazzi]

eharding said:

Example done with PIL/Pillow, using requests to grab a font and mask image....

from PIL import Image, ImageDraw, ImageFont
from random import randint
from io import BytesIO
import requests

background_image = Image.new('RGB', (512,512), color = 0xFFFFFF)

text_image = Image.new('RGB', (512,512), color = 0x000000)

text_image_draw = ImageDraw.Draw(text_image)

word_array = "Powerfully Built Company Directors".split(' ')

font_get_response = requests.get("https://tinyurl.com/4rr7uaed")

ttf_font = ImageFont.truetype(BytesIO(font_get_response.content), size=15)

for _ in range(0,200):
  the_word =  word_array[randint(0, len(word_array) - 1)]
  text_image_draw.text((randint(0,511), randint(0,511)), the_word, fill = 0x00FF00, font = ttf_font)

mask_logo_get_response = requests.get('https://thumbsnap.com/i/2g7fS7iP.png')

mask_logo_image = Image.open(BytesIO(logo_get_response.content))

composite_image = Image.composite(background_image, text_image, mask_logo_image)

Looks good will try this shortly.
Thanks so much for taking the time to do this!
Regards
LB