How to find, remove and read text in this symbol ${} - groovy

Sample data key in by user.
booking/${mm}/${yyyy}
${yyyy}/booking/${mm}
booking/${mm}${yyyy}/00
My problem is how to take out ${ } and read what contain in there and then replace by month/year depend on format.
So the output should be "booking/10/2013" after save into database.
Im using Grails. Hope can solve this problem using Java / groovy.

i just solve the problementer code here
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class Main {
public static void main(String[] args) {
String bookingNo1 = "booking/${mm}/${yyyy}";
String bookingNo2 = "${yyyy}/booking/${mm}";
String bookingNo3 = "booking/${mm}${yyyy}/00";
String newDate = null;
newDate = getDataString(bookingNo1);
System.out.println(newDate);
newDate = getDataString(bookingNo2);
System.out.println(newDate);
newDate = getDataString(bookingNo3);
System.out.println(newDate);
}
public static String getTimeString(String pattern) {
SimpleDateFormat format = new SimpleDateFormat();
format.applyPattern(pattern);
return format.format(new Date());
}
public static String getDataString(String dateInput) {
String dateString = dateInput;
String regex = "\\$\\{(mm|yyyy|DD|MM)\\}";
Pattern pattern = Pattern.compile(regex);
Matcher matcher = pattern.matcher(dateInput);
while (matcher.find()) {
String datePattern = matcher.group()
.replaceAll("(\\$|\\{|\\})", "");
dateString = dateString.replaceAll("\\$\\{" + datePattern + "\\}",
getTimeString(datePattern));
}
return dateString;
}
}

You can do this with Groovy:
// Given these inputs
def inputs = [ 'booking/${mm}/${yyyy}',
'${yyyy}/booking/${mm}',
'booking/${mm}${yyyy}/00' ]
// Make a binding for 'mm' and 'yyyy'
def date = new Date()
def binding = [ mm : date.format( 'MM' ),
yyyy : date.format( 'yyyy' ) ]
// Then process each input with STE and print it out
inputs.each { input ->
println new groovy.text.SimpleTemplateEngine()
.createTemplate( input )
.make( binding )
}
That prints:
booking/10/2013
2013/booking/10
booking/102013/00

Related

How to get list of Named range,sheet name and referance formuls using XSSF and SAX (Event API) for large excel file

I'm tring to to read large excel file (size~10MB,.xlsx) .
I'm using below code
Workbook xmlworkbook =WorkbookFactory.create(OPCPackage.openOrCreate(root_path_name_file));
But it's showing Heap memory issue.
I have also seen other solution on StackOverflow some of them given to increase the JVM but i dont want to increase jvm.
Issue 1) We can't use SXSSF (Streaming Usermodel API) because this is only for writing or creating new workbook.
My sole objective to get the number of NamedRange of sheet, Total number of sheet and their sheet name for large excel file.
If the requirement is only to get the named ranges and sheet names, then only the /xl/workbook.xml from the *.xlsx ZIPPackage must be parsed since those informations are all stored there.
This is possible by getting the appropriate PackagePart and parsing the XML from this. For parsing XML my favorite is using StAX.
Example code which gets all sheet names and defined named ranges:
import org.apache.poi.openxml4j.opc.OPCPackage;
import org.apache.poi.openxml4j.opc.PackagePart;
import javax.xml.stream.XMLEventReader;
import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.events.StartElement;
import javax.xml.stream.events.EndElement;
import javax.xml.stream.events.Characters;
import javax.xml.stream.events.Attribute;
import javax.xml.stream.events.XMLEvent;
import javax.xml.namespace.QName;
import java.io.File;
import java.util.regex.Pattern;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.util.HashMap;
class StaxReadOPCPackageParts {
public static void main(String[] args) {
try {
File file = new File("file.xlsx");
OPCPackage opcpackage = OPCPackage.open(file);
//get the workbook package part
PackagePart workbookpart = opcpackage.getPartsByName(Pattern.compile("/xl/workbook.xml")).get(0);
//create reader for package part
XMLEventReader reader = XMLInputFactory.newInstance().createXMLEventReader(workbookpart.getInputStream());
List<String> sheetNames = new ArrayList<>();
Map<String, String> definedNames = new HashMap<>();
boolean isInDefinedName = false;
String sheetName = "";
String definedNameName = "";
StringBuffer definedNameFormula = new StringBuffer();
while(reader.hasNext()){ //loop over all XML in workbook.xml
XMLEvent event = (XMLEvent)reader.next();
if(event.isStartElement()) {
StartElement startElement = (StartElement)event;
QName startElementName = startElement.getName();
if(startElementName.getLocalPart().equalsIgnoreCase("sheet")) { //start element of sheet definition
Attribute attribute = startElement.getAttributeByName(new QName("name"));
sheetName = attribute.getValue();
sheetNames.add(sheetName);
} else if (startElementName.getLocalPart().equalsIgnoreCase("definedName")) { //start element of definedName
Attribute attribute = startElement.getAttributeByName(new QName("name"));
definedNameName = attribute.getValue();
isInDefinedName = true;
}
} else if(event.isCharacters() && isInDefinedName) { //character content of definedName == the formula
definedNameFormula.append(((Characters)event).getData());
} else if(event.isEndElement()) {
EndElement endElement = (EndElement)event;
QName endElementName = endElement.getName();
if(endElementName.getLocalPart().equalsIgnoreCase("definedName")) { //end element of definedName
definedNames.put(definedNameName, definedNameFormula.toString());
definedNameFormula = new StringBuffer();
isInDefinedName = false;
}
}
}
opcpackage.close();
System.out.println("Sheet names:");
for (String shName : sheetNames) {
System.out.println("Sheet name: " + shName);
}
System.out.println("Named ranges:");
for (String defName : definedNames.keySet()) {
System.out.println("Name: " + defName + ", Formula: " + definedNames.get(defName));
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
}

Excel File Upload with Kendo UI

while I was trying to upload an excel file with kendo ui I found a code on the internet. It is using a keyword named "Constants" but this keyword does not recognize the ".xls" file extension. I am stuck at this and did some research but have no answer to solve this. Here is my code:
public ActionResult Submit(IEnumerable<HttpPostedFileBase> files)
{
if(files!= null)
{
string fileName;
string filePath;
string fileExtension;
foreach(var f in files)
{
//Set file details
SetFileDetails(f, out fileName, out filePath, out fileExtension);
if(fileExtension == Constants.xls || fileExtension == Constants.xlsx)
{
//Save the uploaded file to app folder
string savedExcelFiles = Constants.UploadedFolder + fileName;
f.SaveAs(Server.MapPath(savedExcelFiles));
ReadDataFromExcelFiles(savedExcelFiles);
}
else
{
//file not supported send alert
}
}
}
return RedirectToActionPermanent("Index","Connect");
}
private static void SetFileDetails(HttpPostedFileBase f,out string fileName,out string filePath,out string fileExtension)
{
fileName=Path.GetFileName(f.FileName);
fileExtension=Path.GetExtension(f.FileName);
filePath = Path.GetFullPath(f.FileName);
}
private void ReadDataFromExcelFiles(string savedExcelFiles)
{
var connectionString = string.Format("Provider=Microsoft.ACE.OLEDB.12.0;Data Source={0};Extended Properties=Excel 12.0;",Server.MapPath(savedExcelFiles));
//fill the DataSet by the sheets
var adapter = new OleDbDataAdapter("SELECT * FROM [Sheet1$]",connectionString);
var ds = new DataSet();
List<UploadExcel> uploadExl = new List<UploadExcel>();
adapter.Fill(ds,"Subscriber");
DataTable data=ds.Tables["Subscriber"];
GetSetUploadExcelData(uploadExl,data);
}
private static void GetSetUploadExcelData (List<UploadExcel> uploadExl,DataTable data)
{
for(int i=0;i<data.Rows.Count-1;i++)
{
UploadExcel NewUpload = new UploadExcel();
NewUpload.ID = Convert.ToInt16(data.Rows[i]["ID"]);
NewUpload.CostCenter = Convert.ToString(data.Rows[i]["CostCenter"]);
NewUpload.FirstName = Convert.ToString(data.Rows[i]["FirstName"]);
NewUpload.LastName = Convert.ToString(data.Rows[i]["LastName"]);
NewUpload.MobileNo = Convert.ToString(data.Rows[i]["MobileNo"]);
NewUpload.EmailID = Convert.ToString(data.Rows[i]["EmailID"]);
NewUpload.Services = Convert.ToString(data.Rows[i]["Services"]);
NewUpload.UsageType = Convert.ToString(data.Rows[i]["UsageType"]);
NewUpload.Network = Convert.ToString(data.Rows[i]["Network"]);
NewUpload.UsageIncluded = Convert.ToInt16(data.Rows[i]["UsageIncluded"]);
NewUpload.Unit = Convert.ToString(data.Rows[i]["Unit"]);
uploadExl.Add(NewUpload);
}
}
}
I suspect that the Constants.xls relates to a static class or enum that the original code author is using to hold the .xls/.xlsx extensions.
If you create a constants class something like:
public static class Constants
{
public static string xls = "xls";
public static string xlsx = "xlsx";
}
This would then should help.
If you need any more assistance then please let me know.
edit: Just reviewing the code it seems they are also putting in constant mapping for the uploadfolder location as well so I suspect this is just a static class rather than an enum with application specific details. in a way a bit like using the appSettings within webconfig

String was not recognized as a valid DateTime when convert from string to datetime

I got this Error while try to convert a string to datetime "String was not recognized as a valid DateTime"
My string value : "09/25/2012 11:00:36:156"
Code :
capture = Convert.ToDateTime(newRecord.CaptureTime),
I suggest to You use Regular Expressions. You will have validation input string and robust mechanism of DateTime convert:
void Main()
{
string datestring = "09/25/2012 11:00:36:156";
string regexpr = #"(?x)(?i)
(\d{1,4}) [./-]
(\d{1,2}) [./-]
(\d{1,4}) [\sT] (\d+):(\d+):(\d+) \s? (A\.?M\.?|P\.?M\.?)?";
DateTime capture = new DateTime(); // set to default value in case datestring isn't valid
if (Regex.IsMatch(datestring, regexpr)
&& DateTime.TryParse(Regex.Match(datestring, regexpr).Groups[0].Value, out capture))
{
// convert is succeded
}
else
{
// Handle invalid dateString
}
Console.WriteLine(capture);
}
Or DateTime.TryParseExact(). Read more here.
the problem is with the last colon sign (':') replace with . 09/25/2012 11:00:36:156
var dateString = "09/25/2012 11:00:36:156";
var modifiedString = dateString.Substring(0, dateString.Length - 4) + "." + dateString.Substring(20);
//var modifiedString = dateString.Substring(0, dateString.Length - 4);
var dateValue = Convert.ToDateTime(modifiedString);

How do I create an OrderBy statement using a reflected value?

I would like to create a method that orders an IEnumerable List by a given property where the property is passed into the method by a string i.e. (Mind you the first code example does not work, but the second does and is what I am trying to emulate dynamically).
string sortName = "SerialNumber";
IEnumerable<PartSummary> partList = FunctionToCreateList();
partOrderedList = partList.OrderBy(what do I stick in here);
that would be equivalent to
IEnumerable<PartSummary> partList = FunctionToCreateList();
partOrderedList = partList.OrderBy(p => p.SerialNumber);
How can I accomplish this?
Are you saying you want to pass the order by in to your method? If so, you can use this:
Expression<Func<PartSummary, bool>> orderByClause
Then you can do this:
partOrderedList = partList.OrderBy(orderByClause);
Then you can handle your order by in your business layer or wherever you wish.
Okay, update: If you want to pass in the column name as a string you can do something like as follows:
Create a static class for an extension method (reference: http://social.msdn.microsoft.com/Forums/en-US/linqprojectgeneral/thread/39028ad2-452e-409f-bc9e-d1b263e921f6/):
static class LinqExtensions
{
public static IQueryable<T> OrderBy<T>(this IQueryable<T> source, string sortingColumn, bool isAscending)
{
if (String.IsNullOrEmpty(sortingColumn))
{
return source;
}
ParameterExpression parameter = Expression.Parameter(source.ElementType, String.Empty);
MemberExpression property = Expression.Property(parameter, sortingColumn);
LambdaExpression lambda = Expression.Lambda(property, parameter);
string methodName = isAscending ? "OrderBy" : "OrderByDescending";
Expression methodCallExpression = Expression.Call(typeof(Queryable), methodName,
new Type[] { source.ElementType, property.Type },
source.Expression, Expression.Quote(lambda));
return source.Provider.CreateQuery<T>(methodCallExpression);
}
}
Then you can create your method:
static IQueryable<PartSummary> FunctionToCreateList()
{
IList<PartSummary> list = new List<PartSummary>();
list.Add(new PartSummary
{
Id = 1,
SerialNumber = "A",
});
list.Add(new PartSummary
{
Id = 2,
SerialNumber = "B",
});
return list.AsQueryable();
}
And then call your method:
static void Main(string[] args)
{
IQueryable<PartSummary> partOrderedList = FunctionToCreateList();
PartSummary partSummary = new PartSummary();
string sortBy = "Id";
partOrderedList = partOrderedList.OrderBy(sortBy, false);
foreach (PartSummary summary in partOrderedList)
{
Console.WriteLine(summary.Id + ", " + summary.SerialNumber);
}
Console.ReadLine();
}
Now you can pass in the column name as a string and sort.
Hope this helps!
You can also avoid extending and just use a compiled expression tree to accomplish this:
public Func<T, object> ResolveToProperty<T>(String propertyName)
{
Type t = typeof(T);
var paramExpression = Expression.Parameter(t, "element");
var propertyExpression = Expression.Property(paramExpression, propertyName);
return Expression.Lambda<Func<T, object>>(propertyExpression, paramExpression).Compile();
}
string sortName = "SerialNumber";
IEnumerable<PartSummary> partList = FunctionToCreateList();
var partOrderedList = partList.OrderBy(ResolveToProperty<PartSummary>(sortName));

how reading nutch generated content data on the segment folder using java

I am trying to read the content data inside the segment folder. I think the content data file is written in a custom format
I experimented with nutch's Content class, but it does not recognize the format.
import java.io.IOException;
import org.apache.commons.cli.Options;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.SequenceFile;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.util.GenericOptionsParser;
import org.apache.nutch.protocol.Content;
import org.apache.nutch.util.NutchConfiguration;
public class ContentReader {
public static void main(String[] args) throws IOException {
// Setup the parser
Configuration conf = NutchConfiguration.create();
Options opts = new Options();
GenericOptionsParser parser = new GenericOptionsParser(conf, opts, args);
String[] remainingArgs = parser.getRemainingArgs();
FileSystem fs = FileSystem.get(conf);
String segment = remainingArgs[0];
Path file = new Path(segment, Content.DIR_NAME + "/part-00000/data");
SequenceFile.Reader reader = new SequenceFile.Reader(fs, file, conf);
Text key = new Text();
Content content = new Content();
// Loop through sequence files
while (reader.next(key, content)) {
try {
System.out.write(content.getContent(), 0,
content.getContent().length);
} catch (Exception e) {
}
}
}
}
org.apache.nutch.segment.SegmentReader
has a map reduce implementation that reads content data in the segment directory.
spark/scala code to read data from the segments content folder.
How I read from the content folder in my project.
I have created a case class page which holds data read from the content folder
case class Page(var url: String, var title: String = null
,var contentType: String = null, var rawHtml: String = null,var language: String = null
,var metadata: Map[String,String])
Code to read from content folder
import org.apache.commons.lang3.StringUtils
import org.apache.hadoop.io.{Text, Writable}
import org.apache.nutch.crawl.{CrawlDatum, Inlinks}
import org.apache.nutch.parse.ParseText
import org.apache.nutch.protocol.Content
val contentDF = spark.sparkContext.sequenceFile(path.contentLocation, classOf[Text], classOf[Writable])
.map { case (x, y) => (x.toString, extract(y.asInstanceOf[Content])) }
/** converts Content object to Page **/
def extract(content: Content): Page = {
try {
val parsed = Page(content.getUrl)
var charset: String = getCharsetFromContentType(content.getContentType)
if (StringUtils.isBlank(charset)) {
charset = "UTF-8"
}
parsed.rawHtml = Try(new String(content.getContent, charset)).getOrElse(new String(content.getContent, "UTF-8"))
parsed.contentType = Try(content.getMetadata.get("Content-Type")).getOrElse("text/html")
// parsed.isHomePage = Boolean.valueOf(content.getMetadata.get("isHomePage"))
parsed.metadata = content.getMetadata.names().map(name => (name,content.getMetadata.get(name))).toMap
Try {
if (StringUtils.isNotBlank(content.getMetadata.get("Content-Language")))
parsed.language = content.getMetadata.get("Content-Language")
else if (StringUtils.isNotBlank(content.getMetadata.get("language")))
parsed.language = content.getMetadata.get("language")
else parsed.language = content.getMetadata.get("lang")
}
parsed
} catch {
case e: Exception =>
LOG.error("ERROR while extracting data from Content ", e)
null
}
}
/**Get Html ContentType **/
def getCharsetFromContentType(contentType: String): String = {
var result: String = "UTF-8"
Try {
if (StringUtils.isNotBlank(contentType)) {
val m = charsetPattern.matcher(contentType)
result = if (m.find) m.group(1).trim.toUpperCase else "UTF-8"
}
}
result
}

Resources