Deep Writing Resources by Max De

How to write with artificial intelligence

An easy guide to “Deep Writing” without writing any code

Note: This tutorial only works on Mac, not on Windows

In the past few days, I’ve taught a machine learning algorithm how to write in the style of Harry Potter, Hamilton (the musical), and HBO’s Silicon Valley. The mostly non-sensical, occasionally human-like, topically-flavored writing seems to be amusing not only to me, but to many others.

“Dumbledore will get out from behind a cream cake” — Harry Potter: Written by Artificial Intelligence

Thus, I’ve made this quick tutorial to teach you how to create your own instances of “Deep Writing”. This is not going to be an in-depth description of the underlying technology — but instead, a step-by-step guide that anybody can follow (even if you have no coding or machine learning experience).

Step 0: Understanding the simple intuition

Here is a very crude approximation of what is involved in the Deep Writing process. More than anything, this is meant to give you enough intuition and appreciation to follow along with the rest of the tutorial.

1. You show a computer some sample text (for example, the Harry Potter books).
2. The computer identifies all the unique words in the sample text.
3. The computer groups words based on how often they appear together in the sample text (using a particular mathematical model). This is the “learning” portion of “Deep Learning”.
4. You pick a starting word (for example, “The”).
5. Using what it learned in Step 3, you ask the computer to guess the word most likely to come after the starting word (i.e. “The”). This is recorded as the second word.
6. Then, based on the first two words, you ask the computer to guess the third word. And so on.
7. Eventually, you tell the computer to stop guessing after many words, and you have successfully created your Deep Writing.

Step 1: Download the code

We are going to use code written by Sung Kim (who teaches computer science at HKUST). This code is very similar to the code I used, but is a little bit more generally applicable.

Visit this link, click on the green “Clone or download button”, and then choose “Download Zip”.

Find the zip file in your downloads folder, and double-click to unzip it. Drag the folder to your desktop.

Step 2: Customize the sample text

Open the file input.txt, which you can find in word-rnn-tensorflow-master > data > tinyshakespeare > input.txt. Then, delete all the text in the file, and replace it with the sample text that you want to use. Make sure you save the updated file.

This sample text is the text that your algorithm will read and use as inspiration. For example, you can use the text of Harry Potter, the lyrics to Hamilton, the scripts from Silicon Valley, and so on.

The longer the sample text, the “better” the output will be. Of course, the longer the sample text, the longer it will take to train your model. For comparison, the Harry Potter sample text was 467,678 words long.

Step 3: Install TensorFlow

TensorFlow is a machine learning library made by Google. We need to download it in order to run our code.

To do this, we will use the Terminal. To open the Terminal, click command + space, which will open “Spotlight Search”. Type “Terminal” and click enter.

The terminal should open.

Copy and paste the following line into the Terminal and click enter.

sudo easy_install pip

Then, copy and paste this next line and click enter.

sudo easy_install --upgrade six

These two line prepare your system to install TensorFlow.

Then, copy the below line into the Terminal and click enter. This identifies which version of TensorFlow you want to install.

export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/tensorflow-0.9.0-py2-none-any.whl

Copy this final line and click enter. This will start the installation.

sudo pip install --upgrade $TF_BINARY_URL

After a few minutes, the installation will finish and you can now use TensorFlow on your computer.

Step 4: Train the model

With TensorFlow downloaded on your computer, it’s time to train your model (i.e. “group” words based on patterns).

To start, we want to let the computer know that we want to use TensorFlow. Copy and paste the line below into the Terminal and click enter.

source ~/tensorflow/bin/activate

Now, we want to run the training file. To do this, we first need to tell the Terminal where the file is. Copy and paste the below line and click enter.

cd ~/desktop/word-rnn-tensorflow-master

Lastly, start the training file by copying and pasting the following line into the Terminal. Click enter and the training will begin.

python train.py

Training will take many hours (especially if your sample text is large). It will also drain your battery really fast, so make sure your computer is plugged in. While training, do not close the Terminal window and do not close your computer.

Step 5: Create the “Deep Writing”

Once your model finishes training, it’s finally time to create the Deep Writing.

To prepare, open the file sample.py, which lives inside the folder word-rnn-tensorflow-master. Inside of the file, search for the line of code that says:

parser.add_argument(‘-n’, type=int, default=200, help=’number of words to sample’)

Change the default value to the number of words you want in your instance of Deep Writing. I recommend something between 1000 and 2000.

parser.add_argument(‘-n’, type=int, default=1500, help=’number of words to sample’)

Once you make the change, save the updated file.

Then, go back to the Terminal and copy and paste the following code.

python sample.py

It will take a minute or two, and then the computer will spit out a beautiful instance of Deep Writing.

With a little bit of formatting, it’s ready to be published.

Step 6: Submit your Deep Writing for publication

I created a publication on Medium to collect interesting examples of Deep Writing. If you create Deep Writing that you want to share, Tweet me a link, and I’ll added it the publication.

C Programming Notes

• If a function does not return a value, then its return type should be void. If a function does not use parameters, then its param-list should contain the keyword void.
• All C programs consist of one or more functions, each of which is named  a subroutine that can be called by other parts of the program.
• Functions are the building blocks of C. A statement specifies an action to be performed by the program. In other words, statements are the parts of your program that actually perform operations.
• Although a C program may contain several functions, the only function that it must have is main( ). The main() function is where execution of your program begins. That is, when your program begins running, it starts executing the statements inside main(), beginning with the first statement after the opening curly brace is reached.
• The standard library in C contains functions to perform disk I/O (input / output), string manipulations, mathematical computations, and much more.
• In BASIC or Pascal, operations such as writing to a file or computing a cosine are performed using key words that are built into the language. The advantage C gains by having the as library functions is increased flexibility. Library functions can be enhanced and expanded as needed to accommodate changing circumstances. The C language itself does not need to change. As you will see, virtually all C programs you create will use functions from the C standard library.  
• Another component common to most C programs is the header file. In C, information about the standard library functions is found in various files supplied with your compiler. These files all end with a .H extension. The C compiler uses the information in these files to handle the library functions properly. You add these files to your program using the #include preprocessor directive. All C compilers use as their first phase of compilation a preprocessor, which performs various manipulations on your source file before it is compiled.
• Preprocessor directives are not actually part of the C language,  but rather instructions from you to the compiler. The #include directive tell the preprocessor to read in another file and include it with your program. Notice that the #include directive does not end with a semicolon. The reason for this is that #include is not a C keyword that can define a statement. Instead, it is an instruction to the C compiler itself. 
• A function prototype declares a function before it is used and prior to its definition. A prototype consists of a function's name, its return type, and its parameter list. 
• When the return statement is encountered, the function returns immediately. No statements after it will be executed. Thus, a return statement causes a function to return before its closing curly brace is reached. 

The Rainbox Vision of Graffiti Ten

How cool will it be if the following happens in the future?

• The fact that when you write a short story, if there is an "enough" storyline that your words turn into an animated movie.
• That the main theme of your writing turns into a sketch, an artwork, as if any cartoonist or artist reads your work and depicts it in the art form. 
• That the words you write turn out to be expressed by a movie short-clip, found from sifting abundant data fed into the "movie-realm",  that depicts the "spirit" of your writing.
• That you can doodle into your text-editor or write how you feel and get to read blurbs of poems that otherwise you'd not be able to find. These poems are knowable unknowables -- the poems that you may find out someday and you may love them but you don't know yet. 

Simple and Concise Cover Letter Template from the book - The Google Resume

Dear [Recruiter or Hiring Manager's Name]:

I am interested in the [job title] advertised on [web site or other source]. With a strong background in [list of tangible skills], and [number of] years of experience, I am confident that I can [general problem you can solve].

My qualifications include the following:

• [Desired Qualification #1]: [proof that you have qualification]
• [Desired Qualification #2]: [proof that you have qualification]
• [Desired Qualification #3]: [proof that you have qualification]
• [Desired Qualification #4]: [proof that you have qualification]

I would love to discuss this opportunity further. I will follow up within a [time frame] to confirm that my application was received, and to schedule a time to talk further. 

Sincerely,
Marjuk [ I love my name, so I use it often ]

The Hallmarks of a Great Cover Letter 

1. Tailored (addressing directly to the person involved with the job)
2. Supported with evidence ( proof of your qualification) 

3. Structured and concise

4. Simple, and direct writing 

5. Professional 

Resume Tips from the book The Google Resume

These are some invaluable resume tips that I collected from the book, The Google Resume. All credit goes to the author, Gayle Laakman McDowell.

The Six Hallmarks of Great Resume

• Accomplishment Oriented ("gets things done"):
  
  A great resume enlists accomplishments instead of responsibilities. Accomplishments tell the future employer that you can get things done and you are result oriented. A common mistake that many people do is that they list out their responsibilities in the job versus their accomplishments in the job. Doing so does not say much about one's capabilities to the potential employer. An accomplishment centric example is following:
  
  "Led entrance strategy for Foobar product in China, and successfully persuaded CEO to refocus division on the enterprise market, resulting in a 7 percent increase in profit."
  
  The counterpart to accomplishment oriented resume is responsibility oriented resume, which focuses participation rather than accomplishment. Gayle strongly encourages in favor of creating an accomplishment oriented resume as employers look for doers and winners. An example of responsibility oriented resume will look like following:
  
  "Analyzed new markets and explored potential entrance strategies for China division".
  
• Quantifiable Results: 
  
  The accomplishments mentioned in the resume should be quantifiable. Quantifiable result provides a concrete context for your success, making your accomplishments more relatable and believable. It  also reduces vagueness in your accomplishment claims. If you say, you helped increased sales in the company, but don't say by how much you have increased the sales, there is no context as to how well you have done. An example of resume point that incorporates the strategy of quantifiable results is following:
  
  Original and Vague: "Implemented crash reporter and used results to fix biggest causes of crashes."
  
  This example can be improved in the following way:
  
  Newly quantified version: "Implemented crash reporter and used results to fix three biggest causes of crashes, leading to a 45 percent reduction in customer support calls."
  
• Well Targeted
  
  The resume should not be a one-size-fits-all resume. It should be well targeted for the job or internship that you are applying to and also should be well targeted to the company as well. If you want to tailor your resume for the position and the company need, you need to research about the company and should try to answer the following questions: what are the company's biggest issues and how would my role would impact the biggest issues of the company?
  
  Even if you don't have the experience in solving the exact problems that the biggest issues that the company is facing, you might have skills that will be helpful for the company in solving those biggest issues.
  
• Universally Meaningful
  
  Resume should be universally meaningful in the sense that people outside the domain will be able to understand and relate to the resume contents. Other way to say this is that the resume should not filled with bunch of technical jargons that readers outside the domain might face hard time to relate. If a technical jargon is necessary to convey the meaning, it should accompany an easy explanation as well.
  
• Clean, Professional, Concise
  
  Few points to remember. Fonts ideally 10pt Times New Roman or Arial. 1 inch margin all side. 1 page resume unless you have more than 10 years of experience. Be consistent with style ( number or bullet whatever you pick, be consistent to use it). Take extra-caution to avoid grammar and spelling errors.
  
• Well Structured and Clear
  
  Each time a recruiter takes a look at your resume, she is looking at few things right away. Your education (school, degree, major, and graduation year),  your professional experience (companies, titles, and length of employment) and related technical skills ( for software engineering job-- programming languages you know, technology stack, platforms, and projects). You should make sure that your resume is well-structured into these category and is presented clearly.

What Graffiti Ten Can Become

Graffiti Ten is a personal journal web application that I have co-founded with Reza. Our reason for creating Graffiti Ten was simple. First, we built this product for ourselves. Second, we wanted a private space where we felt we could write without people judging us, as we escape the tyranny of over-sharing that's the feature of this age. Third, we needed confidence that we can build a product. Graffiti Ten seemed to be a project that we can build without breaking much sweat.

In coming product cycles, I want Graffiti Ten to become:

1. The most intimate writing place in the web: It has to become the most personal writing place in the web. The user experience of the service should inspire people to write. The user interface has to be simple and elegant --- well-thought out and catered to user need.

2. A cross-platform writing platform: To make journaling ubiquitous, we need to make Graffiti Ten multi-platform: web (we already have it), mobile (both Android and iPhone), Desktop (it's crucial in the context of offline and online sync) and Apple Watch (market seems ripe given the transition to conversational / audio entry/interaction.

3. An intelligent platform: Graffiti Ten should essentially become a service that understands you, after al it's your personal diary. it should help you to learn more about you. It should show you how you have evolved as a person over the years given that how you have evolved as a person over the years. It should tell you who you write like, what mood you are in when you write.


These are three higher level ideas with Graffiti Ten. The challenge is to break down each of these into more actionable features. It can become so much more than just a personal journal in the web.

Reminder to Self: there is no free will, but inputs influence the trajectory of decision making.

"There's no such thing as free will" ~ The case for a more compassionate world. https://t.co/B7kIhKxZDQ

— Marjuk (@marjukahm) June 3, 2016

Learnings from Marc Andresseen and the Future of Open-Source Game Engines -- What I am Reading Today

The amount of reading that I do each do sometimes astonishes me. I usually do not keep track of what I read. My reading list includes bunch of books from Kindle and iBooks and plenty of blog posts that I find mostly via twitter. I thought it'd be refreshing if I write small summaries of the reading that I do, so when I look back in a day I could use crystallize some of my learnings. Keeping that in mind, I am summarizing from the resources that I have read or listened to:

1. Marc Andreessen's interview on Tim Ferris's podcast:

These are my takeaways from the interview:

• Smart people should build things. 
• "Be so good that they can't ignore you" inspires me and I believe in this. 
• Hedge fund managers are not beholden to their ideas as when they are told where they are wrong, they get excited to hear how so.  
• Value investing versus technology investing are polar opposite in the type of investing that both do. Value investing bets on the future not changing (e.g. Coca-Cola, Ketchup) whereas venture investing invests in future trend and change (e.g. Google, Uber) 
• I should read Steve Martin's memoir  Born Standing Up.

2. Saku Panditharatne's blog post on the importance on open-sourcing in Gaming:

Here's what I have learned: In the gaming / graphics world, open-source is not as popular as it is in the tech world. Examples: 

• For low level graphics APIs, the proprietary library like DirectX is winning against open-source GL in the gaming world. 
• NVIDIA is acting like Microsoft in the early days when MS was against open-source. NVIDIA makes drivers that are required to be reverse engineered by the open source community to make it work. 

The argument is laid by Saku is two-fold:

• This practice of not code sharing has to change and the gaming/graphics community should come together to create open standards and protocols if they want to see VR or gaming DIY to flourish much like it happened for the web. 
• Without the open-source model, there is a high chance that game engines, which are the OS for the VR,  will be too buggy at the absence of the Linus's principle: "given enough eyeballs, all bugs are shallow" 

Meme of the Day:Don't Discount Bernie Yet

With Bernie's chance to secure Democratic nomination getting thinner with each passing day as of May 12, I am still hopeful of some miracle coming along in the Bernie's way. That is why I have found MEME to be funny.

The Illusion of Creating a Masterpiece: Why Writing Is Hard

Why do we write? Writing, in the simplest sense, implies that you want to say something. You want to communicate something that you feel is worth communicating; Without doing it, you feel that you are having an itch, and that itch is not going anywhere unless you say it. You put letters to make words, weaving them back and forth, to create sentences that make sense, that tell what you have been intending to tell, all along; Your ideas live since then. You give it a form.

How do our ideas go on to live? We use canvases. We station our ideas in playgrounds that are shared, that can be accessed by others to read, like, dislike, enjoy, debate. Once the writing is done, we feel settled: the itching is gone. We get rid of the stranglehold of our words, feeling that we brought justice to the truth-- what we initially set out to say all along. In between all of the process, we feel anxious as to if we are making the right choices with words, tones, length, judgment, and details. We depict pictures into readers’ mind. We think about these pictures. We reread, reexamine, and reemploy our senses to determine whether our initial hunches about our message got reflected into what we wrote.

We also worry about the medium where our writing lives on, as if we feel failing to choose the right medium will cost us the message. Our precious hunches might not be communicated well. We worry that our emotions will go unnoticed, our intentions will get misdirected, and our important little truths will be misinterpreted.

In between the details of writing, the relationship dynamics between you, me, us, and them are complex. We want to make sure our writing reaches the right audience, helps them connect, and lets them reciprocate with us in some ways. For this reason, as writers, we sweat on the details, making sure our words don’t carry racial tone, employ judgments that might hurt the feelings of others, or simply, fail to tell the truth. We sweat over political correctness, religious interrelations, and the power dynamics that our writing contain. This ordeal takes a toll on us, putting our cognitive load to fullest. As writers, after all, we all want to be loved. We worry about instruments that might cost us love. It’s a scary line we want to walk on when we sit to write.

Now, as it may happen to almost to many writers, that we set out to say initially, got never said. We find that our ability to employ words to tell the truth is not enough. There are plenty of rooms for errors. We may have employed wrong words, organized the critical pieces incoherently, rambled on unnecessarily, or simply, failed to deliver a punch line when it was due.

Realizing that now we are stuck between we want to tell the truth and we devised the story sub-optimally , we debate whether we should publish it. What might be called as the great dilemma of being a writer. There are so many scopes for improvement, leaving us with a feeling that the difference between creating a great writing and creating a not-so-great writing is us. We are not impressed by us, considering our shortcomings to deal with lacking good sense, measured tones, and omission of bad words.

We retreat back to inaction, believing in the lie that we were never set out to tell the story. All we had few hunches, the random itches of senses that mind produced when we thought we had something to say but we somehow ruined saying it.

That I feel is the great tragedy of the commons: emotions, story clues, truths, and important hunches going unnoticed, unscratched, and lost in the vast deep ocean of sensory illusions.

Why I Love Economics

I started out my undergrad journey studying economics. I still remember those joyous nights of reading Mankiw, learning Poker, dwelling in The Economist website, and day-dreaming about becoming next John Nash. I still remember that joy I felt when I learned the insight about human behavior that people respond to incentives. It makes so much sense. Such a great combination of beauty, truth, and elegance is wrapped within one principle.

More than anything else, I loved how economics taught me how to think about life. For example, we face trade-offs: we can't have everything we want. That if we want something, we have to think about the price we want to pay for it. And the price of something is what we are willing to give up to get it. It can be time (e.g. investing in relationship). It can be hours of work that we put to reach that magical score to obtain admissions into our dream colleges.

Perhaps, the most elegant among the economic principles to me is the one that says rational people think at margin. Margin means edge: that we always weigh in the benefit that we will get for one extra unit in exchange of the cost of that unit. For rational people, marginal cost-benefit analysis needs to make sense; otherwise, they shouldn't move forward with the decision.

Don't we ask al the time that "is it worth it?" Exactly, here we employ the thinking at margin principle. It's really powerful if you get to apply it in most of your decision making processes. I hope I apply it more actively and more often. I hope you do it as well.

Besides, sharing why I love economics, I want to make a small point. Much of what I write or am going to write occasionally is my attempt to produce more as I want to balance between my consumption diet and creation habit. I am hoping to write more, mostly summaries of readings that I do or my opinions of things that I read or think. I very much hope to receive feedback from you.

Ratio Thinking - An Approach to Life

Just now I came across reading an article by Joel Gascoigne, the founder of Buffer. In this article, Joel discusses about how the ratio thinking shapes how he thinks, operates, and runs his business. It's an an interesting way to approach life. Ratio thinking is based on the "law of averages" as explained by entrepreneur and author Jim Rohn, who said:

"If you do something often enough, you’ll get a ratio of results. Anyone can create this ratio."

That means, if you want to increase achieving the number of your intended successes, you need to attempt more. More attempts, done with creative adjustments, will yield in more intended outcomes. For me, increasing my ratio means taking those chances that I am uncomfortable of taking: sending those emails that I fear of getting no response, asking for those helps that I feel will not be returned, and many more. The sooner we realize the power of ratio thinking, I believe the better are our chances for winning.